The OS is swapping the least recently used memory portions out to disk. In a situation where memory is low, swapping can affect the memory used by the Session View clips, something which is more likely to happen when the clip has not been played for a while. Live pre-loads the initial part of the samples in the Session View to RAM for immediate availability. This is necessary because, unlike clips in the Arrangement, the program cannot anticipate when you will play Session View clips. When the OS "swaps", and a Session View clip is launched, Live has to wait for the OS to bring in the audio from disk, which takes longer than real-time allows. The best remedy for this problem is to close other applications that consume RAM or to put more RAM in your computer.