I have lots of manure. I use drip irrigation. There are several places between rows that get quite a bit of water, so are pretty moist. Would there be a variety of worms that would adapt easily to this garden environment? I know Red Wigglers won't.
If there is enough manure and it stays wet, then I do think red wigglers will live just fine. I know people often say red wigglers cannot make it in a garden, but I find them in my garden frequently. My garden is well watered and covered in organic matter though.
I do need to point out though that the red wigglers would not be able to survive a drought like condition for long without watering the garden and worms.
Alabama Jumpers are the best worms for your garden. They are deep burrowing, powerful soil worms and will turn even the worst clay (like here in AZ) into a fine loamy material.