This was inspired by the recent teardowns of some LED candles with the moving flame effect. This version is super easy to build yourself and has no real electronics other than an LED, resistor and battery. You could even base it on an existing solar powered garden light.
It's basically an LED supported by some stiff insulated wires and with an extra bit of thin copper wire soldered to one of the LED legs and bent in such a way that it supports and centres a small white plastic flame shape so that it wavers freely in the wind. The LED is angled towards the fake flame and with a standard lensed LED a flame shape is effectively projected onto the plastic.
The hole in the plastic flame should be quite small and very clean to allow it to waver back and forth and also twist from side to side for an extra visual effect.
You can experiment with the flame shape. It's easy to cut out of any stiff white plastic you can find (or clear plastic painted white?) and you can either get fancy with the shape or keep it simple, since the main effect is created by the projected beam of the LED. For a faster and heavier flame you can keep the counterweight side shorter and add something heavy like a small screw, washer or bit of wire. Extra weight makes the flame more stable in strong wind but may make it less sensitive to light wind.
Initially I kept the flame support wire quite high at the back so I could try on different flames, but once you've chosen a flame shape you really have to cut it quite short at the back and curl it round on itself so that strong wind doesn't just blow the flame right off the wire.
Consider using a warm white LED for a realistic flame, yellow or orange for a more vividly coloured flame or weird colours like green, blue and red for psychedelic flame effects.