Sounds like the syncro for sure mang...
The small inner teeth on the syncro, over time, wear out (become rounded) and a few bad shifts at high RPMs might actually snap off one of the teeth. This need to be addressed.
In terms of replace or rebuild....I vote for rebuild. B-series syncros are in pairs except for 5th. Meaning, if your fourth gear sync is shot, you will need to replace 3rd and 4th. The part will cost you around the $150-$200 US. While your in there, you might as well replace some bearings and get a new clutch if you were thinking of doing one eventually (heres your chance). But the labour for tranny work is always expensive.
I did the 3+4 sync, replaced EVERY bearing in the tranny, new clutch disk and topped it up with REDLINE tranny fluid this past summer and spent (with tax and labour) around $1600 cdn. You dont have to go as wild as that, but I figured what the fuck...save on labour if you do everything at once.......
And REDLINE tranny fluid is the absolute only thing that will ever be put in my car...period. Give it a shot...
peace.