The "pin" you see may be the oil sprayer. It doesn't push back in. You might as well bite the bullet and pull the pan, crank pulley, fan and timing cover off. I don't know of any other way to do it. If it's a high-mileage car you might consider putting a new timing kit on it (sprockets, chain, tensioner, etc.) at that point.
Are you saying you did this 5 years ago and the car hasn't run since?
Are you saying you did this 5 years ago and the car hasn't run since?