God gives us His joy, comfort, and encouragement when in His wisdom He knows we need it.....and not in such great measure that it spoils us. We can have songs in the night. But we are not here to feel good. Jesus is thoroughly winnowing His threshing floor.....being threshed doesnt' feel so good but we need it. When we are ready the Lord begins to shine His light so that we start to see just how thoroughly sinful we truly are......the goal I believe is that we would thoroughly repent of our sinfulness (old man) in a wholesale way, lock, stock and barrel. We are not to rest on our laurels and use grace as a license to hold onto our sinfulness once the Lord reveals it to us. Like the children of Israel the wilderness journey revealed where they still lacked. It's not necessarily that we sin outwardly, at least not in big ways hopefully, but it more concerns our inward sinfulness, which sometimes causes us to sin inadvertently or without realizing it and which can be sins of commission or omission.
When I came to the Lord the only sin I repented of (shamefacedly) was the sin of unbelief......some other sins fell away automatically.....others I turned from as I was instructed in the word and it wasn't difficult by His grace.....His yoke is easy/light. As time (many years) went on the Lord let me know that bridal intimacy involves letting the Lord into areas of our heart that we would rather keep covered.......and He gave me a dream to let me know this was His great pleasure to cleanse me so that I would not be afraid of this process. We tend to want to cover over and hide our shortcomings rather than first of all face them ourselves and then confess them to the Lord.
Suffering is a key part of the process......the bible says if we suffer in the body we are done with sin.
Heb 12:3-11
For consider him that endured such contradiction of sinners against himself, lest ye be wearied and faint in your minds.
Ye have not yet resisted unto blood, striving against sin.
And ye have forgotten the exhortation which speaketh unto you as unto children, My son, despise not thou the chastening of the Lord, nor faint when thou art rebuked of him:
For whom the Lord loveth he chasteneth, and scourgeth every son whom he receiveth.
If ye endure chastening, God dealeth with you as with sons; for what son is he whom the father chasteneth not?
But if ye be without chastisement, whereof all are partakers, then are ye bastards, and not sons.
Furthermore we have had fathers of our flesh which corrected us, and we gave them reverence: shall we not much rather be in subjection unto the Father of spirits, and live? ( submit to Him in this process until we are thoroughly submitted to Him)
For they verily for a few days chastened us after their own pleasure; but he for our profit, that we might be partakers of his holiness.
Now no chastening for the present seemeth to be joyous, but grievous: nevertheless afterward it yieldeth the peaceable fruit of righteousness unto them which are exercised thereby.
Wherefore lift up the hands which hang down, and the feeble knees;
And make straight paths for your feet, lest that which is lame be turned out of the way; but let it rather be healed.
Elsewhere the scripture says "if we suffer with Him, we will reign (overcome) with Him". There is a lot written about suffering which we tend to kind of gloss over until we are ready to face it. Maybe it is the Lord kind of hiding things from His children when we are not yet ready for them, and then begins to feed us with those things once He deems we are ready.