The check valve is located on the fuel pump outlet.
Your fuel pump should not come on with the key in the on position, only in the start position. My car just went through this a few days ago.
Remove the spade connector from the starter. Turn the key to the on pos. Does the fuel pump run (it should'nt). Now turn key to start pos. Fuel pump should run as long as the key is in the start pos.
Remove the relay box located on the passenger fender well. On the top row (inside to outside) is the fuel pump control relay, fuel pump relay, efi relay.
The f.p. control relay will have a six pin connector with only five wires going to it. Test that relay, per the FSM (factory service manual, if you have one... if not, better get one!). The test info is in the engine fuel section. You will need a digital multimeter and a few alligator clip test leads. Continuinty tests need to be performed on the relay pins with out voltage going to pin 95 (+) and 94 (-). Follow the FSM tests... but on pin 95 to 96, the meter will not beep but will show resistance, which is good. Then you apply 12volts dc to pins 94 and 95 and perform the same test, then ground pin 96 and test again. Look at the flow chart in the fsm, it will explain everything.
The oil pressure switch and alt send a signal to the f.p. control relay. It is a safety circuit so when the car stalls, accident, ect the relay will sense no oil pressure, or alt output and shut off the fuel pump.
Check all grounds too along with the oil pressure switch connections.