Yes, please do. Children are welcome in Hallgrimskirkja. During the mass on Sundays 11 am there are also family programs in the hall of the southern wing. And there are special programs for infants and their families on Wednesdays 10-12 am. For other programs for children ask the staff in the church.