Not sure if this is by design, but it seems that labels are shared across workspace and when you create a new column view for a folder, it will create columns for each label even if they are not assigned within that folder (and therefore empty columns).
The attributes (in this case label) are a part of a Board. They can be re-used between folders and items, or they can be ‘hidden’ from the item. If you want to remove an attribute from an item, you can just hover over that attribute in the sidebar, click on three dots and click remove. This will hide this attribute from a folder, but it will not delete it from other folders.
If there’s no item with that label value, the column will be empty.
However, the column will be shown (even if empty) because you maybe want to move an item to that label value.