Put it immediately after the invitation is shown, so the refusal answers it directly. Follow it with Sad Violin for mock disappointment, or with Wrong Answer Game Show when the invitation itself was a bad idea.