Heat stress adversely affects the litter traits of Landrace, the primary dam breed in Taiwan. This study aimed to evaluate the effects of heat stress during sow pregnancy with homogeneous and heterogeneous residuals to estimate the genetic parameters of the Taiwan Landrace. Performance records for 11 657 litters and weather data from 2008 to 2021 were collected. The climate effect was defined proportionally from cool to hot for climates 1 to 5. The homogeneous residual analysis showed that the heritabilities of the total number born (TNB) and number born alive (NBA) were 0.164 ± 0.014 and 0.111 < ± < 0.014 with residual variances of 10.338 and 9.164, respectively. The heterogeneous residual analysis showed that the residual variances for TNB and NBA were 8.934–11.113 and 8.196–9.810, respectively. For TNB, the residual variance in the herd–year–climate effect differed significantly in climate 1 from climates 3 to 5 (p < 0.01). In NBA, the residual variance was significantly lower in climates 1 and 2 than in climates 3 and 5 (p < 0.01). In conclusion, heritability was estimated for TNB and NBA. In addition, residual variances could interact with the climate effect in heterogeneous residual analysis.