Why does job displacement, e.g., following import competition, technological change, or economic downturns, result in permanent wage losses? The job displacement literature is silent on whether wage losses after job displacement are driven by lost firm wage premiums or worker productivity depreciations. We therefore estimate losses in wages and firm wage premiums. Premiums are measured as firm effects from a two-way fixed-effects approach, as described in Abowd, Kramarz, and Margolis (1999). Using German administrative data, we find that wage losses are, on average, fully explained by losses in firm wage premiums and that premium losses are largely permanent. We show that losses in wages and premiums are minor for workers displaced from small plants and strongly increase with pre-displacement firm size, which provides an explanation for the large and persistent wage losses that have been found in previous studies mostly focusing on displacement from large employers.