I don't think that we should be answering questions that are off-topic. Of course, there may be some disagreement about whether a particular question is off-topic.
Personally, if I think that a question is off-topic, I will flag it (I don't have the rep to vote to close yet), but I won't post an answer to a question that I consider to be off-topic. I am also reluctant to upvote answers to off-topic questions (though sometimes I do upvote such answers when they seem particularly good). I usually leave a comment when I flag a question as off-topic, and if I think that the poster might be able to salvage the question I try to nudge them in a productive direction. If I see that the question has been fixed through edits I will retract my flag, or occasionally I just change my mind and retract the flag.
Sometimes I will answer a well-meant, but off-topic (in my opinion) question in the comments. I am a little ambivalent about this; on the one hand I feel like I shouldn't be providing an answer at all as this just encourages more bad questions, but on the other hand it is hard to stifle the desire to help someone with such a question. Answering with a comment rather than an actual Answer seems to be a compromise, but I don't always feel good about it.