Rain is probably the most obvious enemy to the plein air painter. But a gentle mist can add texture and prolong the dampness needed for a wet-in-wet painting.
Wind is more sneaky. The supporting board that holds the paper acts just like a sail. Many's the time I've walked away from the easel to see if the pattern is working, only to hear the easel crash from a big gust of wind.
Bugs! Once while painting with my Wednesday painting group in Boothbay, I was in a field. Several of my painter friends were at the edge of the field. Suddenly a cloud of corn ants began swirling around my head and dropping into my hair and onto the wet palette. Corn ants only live for a day, but they found me! Later my buddies said I looked like Pig Pen in the Peanuts cartoon strip, with a moving black cloud above my head.
Sun. If you paint with the sun on the paper, the paint will dry too fast and the glare will make you snow blind. Find shade or position your paper so that it will make its own shade.
Tourists can be the worst hazard, especially if you're a beginner or self conscious, or trying to concentrate at a particularly crucial stage of the painting. Little kids are the worst. Often they will point to something in the painting and then walk up and touch it. And answering the same questions from curious bystanders can be a distraction.
So with all these challenges, why paint outdoors? Because..... the sounds, the smells, the breeze, the feel of sun on your skin, the funny stories all tend to make their way into the painting. Just looking at a painting years later, I can recall all the things that happened that day. And if you get an interesting tourist and have a great conversation, you might just make a sale!