It's well established on SE sites that off-topic questions should not be answered. For example, see
Nevertheless, we, as a community, do both not infrequently.
Is downvoting an answer to an off-topic question an appropriate way to discourage the practice?
If a question is off-topic, it should be closed and deleted. There is a voting procedure for that. If you downvote an answer just because you believe it's off-topic, you override the close-voting procedure.
Should comments or posts answering off-topic questions be flagged for deletion?
If a question is off-topic, there is no point to keep the question. The whole question should be deleted.
The line between good and bad question is difficult to recognize. The close-voting procedure allows for somewhat objective judgment. Moreover, it appears to me that the answer voting system is meant to point good and bad answers, rather than to discipline the users. The reputation loss from receiving a downvote is very small.
Downvoting an off-topic question/answer seems a waste of everything. It costs the dv-er, and if (as is often the case) the OP has a rep of 1 or so, the dv doesn't get represented anyhow.
Off-topic questions need deleting. There's no good reason they're there, surely? Which then means the answers (and questions) are superfluous.
Answering dupes, which may get deleted, is a different issue. Sometimes, the question spawns a slightly different slant on a problem - so the question isn't, as may be construed by some, an exact dupe. Thus, answers may well augment those in the other 'dupe'. I agree that an answer which adds nothing more deserves something, either dv or deletion.
Difference between closing and deleting? There's a possibility that those reading the closed questions begin to understand the rationale behind the closure. Deletion avoids this, but 'saves some space'.
If an offtopic post gets answers there are two things that become problematic:
The presence of the question encourages others like it, either through search engines sending more traffic, or simply from folks thinking those sorts of questions are okay, as they get answers.
Upvoted answers make it harder for questions to be deleted, even if the questions are heavily downvoted. Normally automatic processes remove very bad posts.
I will usually downvote answers to obviously off topic questions before voting to close the question.
Personally, I have not been systematically flagging or downvoting these answers. I don't feel that answers written in good faith should be necessarily downvoted; however, "in good faith" here would mean not only that the answer is otherwise acceptable, but that the answerer reasonably believed that the question was okay.
For example, if the question is marked as a duplicate but the answer is substantially different than the linked post, my current logic would not fault the answerer. If the answer is different, that is evidence enough to me that this was written with the belief that this question is not a duplicate. I don't mean to say that intent should be evaluated in a merit-based vote system, but I recognize that there are gray areas with some duplicates.
For off-topic questions, I think it's important to note that even experienced users can disagree on the validity of questions every once in a while, and if I think they may have a legitimate argument for its topicality, I would be okay with that answer as well.
Most cases, though, are night and day: either on- or off-topic, unique or duplicate. When a question clearly should be closed, I will often hold answers to that question in some contempt. Generally, those are bad-faith answers, and while I don't have a hard rule, it doesn't take much for that kind of answer to get the down arrow from me. A flag is an available option, but honestly it's worth considering a comment. That might be the better option in a lot of cases, particularly with newer users.
Happily, most black-and-white SE questions don't attract any answers (Yay community!). So when a question IS being considered for closure and you believe it should be open,
Answering is possibly a bit presumptive if others are indicating that they strongly believe the question is a duplicate or off-topic, so use good judgement in that regard.
These are just my current opinions on the issue, and I would be happy to hear other ideas.
