Eastern oyster (Crassostrea virginica) populations are declining due to habitat degradation, overharvesting, and disease. One factor of potentially equal importance that has received little focus is the impact of food quantity and size composition of the seston supporting larval development, survival, and success at metamorphosis. Phytoplankton serve as a primary source of nutrition for oyster larvae; its size composition and content tend to fluctuate with environmental conditions. These fluctuations, in turn, may significantly impact larval performance. To investigate the potential of variations in the size composition of the seston determining the dynamics of larval performance, simulations are developed to project larval success in relation to variations in three size classes of planktonic food, <5, 5–20, and >20 µm. A biochemically-based larval performance model is implemented to track key outcomes, defined as the success rate at metamorphosis, mortality during development due to suboptimal food supply, and mortality due to egg quality characteristics supporting early development. The results support the potential of variations in the size composition of planktonic food as a key driver of outcomes, potentially explaining some of the well-described year-to-year volatility in recruitment. The results point to a number of key influences on larval performance including the critical need for sufficient availability of small food particles at the end of the trochophore stage, suggesting a potentially key match–mismatch point in larval development. The results point to the influence of larger food particles later in larval life in supporting increased size at metamorphosis and in reducing planktonic lifetimes, thereby reducing exposure to predation. Simulations also show the influence of egg size and larval physiology supporting increased survival when food size composition is suboptimal. Given the outcomes of this study, increased interest in the size-composition of seston in the field relative to larval performance metrics would seem warranted.