The origin doesn’t include actually having sex with a dog though. “Fuck the dog” was just an expression for being lazy and “screw the pooch” simply makes it more palatable.
Whether the action was feeding, walking, or fornicating, though, all of these early examples were used to mean “to loaf around” or “to waste time” (dogs have often been associated with laziness, as in the expression “dogging it”). Later on, possibly around World War II, “fucking the dog” and its euphemistic equivalents took on a secondary meaning of “blundering.”