Most likely the box.
The clutch would affect all shifts and would be most noticable when the car is stationary and you are trying for reverse.
You could check your tranny oil level but if it is low then the damage may be already done. Maybe new oil might help a little bit. I had good results putting synthetic tranny oil in an old RX-7 tranny.
It's probably not your clutch. It would exhibit signs of slipping, etc, but no grinding. Filling, replacing the fluid in your tranny is a good idea and may improve it some. If the damage has been done you can rebuild it or swap in another. With our z31's it's not terribly difficult to swap another one on and there are tons of people on the z31.com classifieds who would love to sell you one at a reasonable price.
I would try replacing the fluid as Mister_T suggested first!
i had a similar problem , it came out to be a plastic bushing that goes from the end of the shifter and connects to the tranny , it had came off so it hasd play and it had problems going into second , but this is in a n/a tranny, and depending on the grinding im guessing it could be something more serious in the tranny
my car does the same thing. My pressure plate has collapsed slightly on one so it never fully releases, even with the clutch adjusted all the way out. I am going to put an ACT clutch in to replace it in a week or so, but my synchros are no doubt shot to **** and the ends of my gears are no doubt ground down due to clashing of teeth when i shift. I was considering rebuilding my tranny while its out, but since i've never done work on a tranny maybe i should just see if it really needs it first.