Neither Thornton and Gifford nor Dent so much as mention this expression. Documentary evidence outside Ulysses shows two different meanings for the catch phrase:
a) the meaning which is still current in Australia: “an old way of saying farewell” (John Miller, The Lingo Dictionary: Of Favourite Australian Words and Phrases, 2011), and
b) the meaning probably intended by Zoe and known in Ireland as evidenced by Padraic O'Farrell in How the Irish speak English, 1993, p. 12:
The offhand nature of Wexford people gave rise to two peculiar sayings during the Second World War. Scarce commodities were passed around then and great thanks would be bestowed for the gift of an ounce or two of tea. The donor would shrug off the kind deed by saying, "Ah thank your mother for the rabbits, the soup was lovely!" It was a longwinded, "Ah, for nothing!"
The earliest source so far seen is from Chambers's Journal of 1917:
Get the kettle boiling, and we'll have some toddy when I get back, and "thank your mother for the rabbits!" This last an insane remark, but quite like Smiler. He was off on a four-to-one chance of ever seeing the regiment again. (p. 447)
It is hard to say what the “inane remark” means here. Similarly ambiguous is the next example:
"Thank your Mother for the rabbit", and "the bottom stair's the lowest", are for ever in her mouth. Her circumlocutions have become second nature.
James Agate Blessed are the rich: episodes in the life of Oliver Sheldon (1924), p. 202
This Australian newspaper source seems to indicate that the expression may have customarily have been followed by “it was (or they were) lovely”:
Able Seaman Jobbings's request to the Regulating Petty Officer to "Thank mother for the rabbits, they were lovely."
1940 Argus (Melbourne, Victoria) (Weekend magazine) 6 January, p. 8
It is interesting to see that Eric Partridge, too, faced problems when commenting on the phrase in his Dictionary of Catch Phrases American and British (1985) p. 294:
"thank your mother for the rabbit" was brought to my attention by the late Frank Shaw1 in 1969, but without definition, date, milieu.