If you went somewhere to eat lunch, then you went TO lunch.

If you went somewhere to get lunch and bring it back, then you went FOR lunch.

If you went somewhere to pray, then you went TO prayer.

If you went somewhere to have someone else pray for you, then you went FOR prayer.

“Gone” is a past participle and is always used with the helping verb “have/has”.

You went to lunch today. You have gone to lunch every day this week.

You went for lunch last Tuesday and brought back food for everyone in the office. You have gone for lunch every Tuesday this month.

He went to prayer yesterday. He has gone to prayer every Sunday this year.

He went for prayer last month. He has gone for prayer every time his church has had a healing service.

Gone to lunch–you left and will eat before returning

Gone for lunch–you’ve left and will be back with lunch

Same thing with went only it’s past tense. You’d use went to describe what you’ve already done.

whether went or gone.