This advice is for paddling on sheltered waters (Thames above Teddington or canals). The advice would be different for white water or sea kayaking!
There is no reason not to paddle in winter; the old quote by Alfred Wainright that “There’s no such thing as bad weather, only the wrong clothing” applies to kayaking (most of the time).
To keep warm whilst paddling, the layering principle works well: Baselayer, midlayer and shell. Your baselayer needn’t be that warm, it’s there for wicking moisture away from your body. The mid layer keeps you warm and the shell (cag) keeps you dry and keeps the wind off. Most importantly, you shouldn’t wear any cotton at all. If it gets wet, you will get very cold. Synthetic layers or thin marino wool is good.
Warm footwear is also important. I find thick (5mm) neoprene booties work well. You might also want to consider waterproof socks such as those from Sealskinz or Reed Chilcheater.
For hands, I find pogies to be much more effective than gloves for paddling. You still have good contact with the paddle but they also keep your hands very warm. There are several pairs in the women’s changing room.
Don’t forget that spraydecks really do help to keep you warm and dry as well, again there are several in the women’s changing room that fit our K1s and K2s.
It’s a very good idea to carry spare layers in a drybag in your boat, just in case the worst happens and you take a dip. The key to staying warm is to stay dry, so if you fall in, a quick change will save the day. Some people also carry a bothy jacket that you can put over everything including a BA. This adds several degrees warmth. If you’re on a trip, it’s also worth taking a thermos with hot drink.
Needless to say, always wear a buoyancy aid with whistle and carry a fully charge mobile phone with you in a waterproof case.