Art is no longer just paintings in galleries, if it ever was. Art, both endorsed or disruptive, ranging from innocuous to offensive, is on the streets and largely unavoidable.
Not making the best use of your time doesn't make you selfish. Creating art is beneficial to people for different reasons. Is helping out at the food bank selfish because I could be spending my time constructing houses in Central Africa?
Spending 100% of your time on 'generous uses' may actually produce less good for all in the long run. If you go without eating in order to help others, you will eventually starve to death and be unable to help anyone. A similar argument may be made that leisure, such as art, prevents burnout.