Although our visual experience of the world is rich and full of detail, visual short-term memory (VSTM) can retain only about four objects at a time. Long-term memory (LTM) for pictures lasts longer but may rely on abstract gist, raising the question of how it is possible to remember details of natural scenes. We studied the accumulation and persistence of memory for pictures shown for 1–20 s. Performance in answering questions about the details of pictures increased linearly as a function of the total time that the scene was viewed. Similar gains in memory were found for items of central and marginal interest. No loss of memory was found for picture detail over a 60-s interval, even when observers performed a VSTM or reading task during the delay. Together these results suggest that our rich phenomenological experience of a detailed scene reflects the maintenance in memory of useful information about previous fixations rather than the limited capacity of VSTM.