A definition of “sport” as “an activity undertaken for enjoyment, in a specific environment, commonly outside” is pretty decent in my view.
The field - outdoors, countryside, perhaps an actual field, though not necessarily.
As you say though, both are plausible, and that ambiguity is part of what makes the phrase so interesting.
However, this shows why this dictionary is not actually good for modern, practical use, IMO.