For our heart shall rejoice in him - See the notes at Psalms 13:5.
Because we have trusted in his holy name - In "him," the "name" often being put for the person himself. See the notes at Psalms 20:1. The idea is:
(a) that the fact of our having put our trust in God is in itself an occasion of joy or rejoicing;
(b) that the result will be joy, for we shall never be disappointed.
It will always, and in all circumstances, be a source of joy to anyone that he HAS put his trust in the name of God.
DISCLAIMER: Church of the Great God (CGG) provides these resources to aid the individual in studying the Bible. However, it is up to the individual to "prove all things, and hold fast to that which is good" (I Thessalonians 5:21). The content of these resources does not necessarily reflect the views of CGG. They are provided for information purposes only.