Colony size is the area that your colony covers in square kilometers.
The space it requires depends on number and
wealth of inhabitants, government buildings and its mix of
land development.
While land has value and as such acquiring more does not directly reduce your colonies
wealth, there are administrative costs involved in both buying and selling land. Large variations, i.e. when building a lot of farms at once, can have a temporary but large impact on the colony's wealth and the associated effects.
See also:
Density