Throughout the game you will be set upon by Barbarians, this can happen at any time on any of your cities or captured tiles. Luckily the news feed gives you a warning and how long you have to prepare. If they successfully raid your city, a building will be set on fire. You can extinguish the fire by clicking on it.
The Barbarians may attack your luxuries or settlement more frequently if they are attacked directly.