Left for (the) office

[Brahman]

Could someone help me with the following?

Say, I am lunching in a restaurant. A colleague asks, "Where are you going after this?" Should I say, "(I am) returning to the office" or "(I am) returning to office."

Is the definite article, in such a situation, optional?

What about if I am going to work in the morning --- should it be "I am going to the office" or "I am going to office"?

[Josef]

In all cases "the office" (or "my office" or "his office" etc). "I am going to office" is not English. This applies to most locations - the station, the park, the airport etc. But note that there are few exceptions (which are established expressions), eg:

go home
go to school