That is caused by the recording device mostly. Although there is always some background noise, the majority is caused by:
1) the pickup device or microphone used.
2) the OpAmp electronics within the recorder or playback device.
3) the media on which its recorded (analog recorders only)
4) lack of proper shielding in components.
Analog tape units although I believe to be more prone to EVP's are plagued with very high signal to noise ratio. Noise is induced by the analog OpAmps, also by the tape medium, and again by the playback device.
Digital recorders are plagued by noise from the digital circuitry, since it uses a high frequency clock source to sequence the controllers in the recorders, if its not properly shielded or suppressed from the analog circuits, it can result in a great deal of background noise.
Always look at the signal to noise ratio when purchasing EVP equipment.