Part of the answer lies in the spacing of the plants relative to the width of the beds. With intensive cropping methods, growers will often put a high number of plants into the beds, and this increases the amount of root competition for space in that horizon of the soil. There’s little competition below them, and often plenty of nutrient and water due to the way the beds are constructed, so it’s the optimal path for the plants’ roots to take.

Sometimes factors like crop rotation come into play, such as when growing carrots, parsnips, or other taprooting vegetables before another crop (sometimes seeding the new crop before pulling the root crop). The deep roots put sugars into the soil which cultivate a host of microscopic life, and the second crop can sometimes use these colonies, or can at least make use of the crumbly soil structure left over from the first.

Not a stupid question at all! This method works for several reasons:

• Typical soil contains the majority of nutrients in the topsoil layer, which is roughly 1-10in (2.5-25cm) deep. Because compost is added up to 2ft (60cm) deep, roots will grow much deeper to take advantage of the easily accessible nutrients.
• When the soil is kept loose, plants can grow roots through it much easier; French intensive gardening focuses on loosening the soil very deeply and avoids compacting it so plants don’t have to expend as much energy pushing their roots through hard soil.
• As a result of plants growing roots deeper rather than wider, they’re able to be planted much more densely and grow more produce than market gardening in the same size plot!