You were the teacher and they were the students. Different dynamic. When we've broken away from a group we always informed the tour leader and we knew when to be back if necessary. I remember in Hue, Vietnam, the group was going on some kind of day tour. We were going to a restaurant that Tony Bourdain said had "the best soup in the world." We were there at 8AM when she started serving and when it was gone that was it for the day. Our day in no way interfered with the group.