When doing group by the number of records shown per group is the same number of records per page in list view and cannot be changed.
No cause as this is by designed of the platform.
The workaround would be to change the number of records shown per page in list view if more records need to be seen per group when doing a group by.
By default the maximum number that can be set is 100 but the system property "glide.ui.per_page" can be modified to add in more values greater than 100.
Please note however, that adding a higher value and using it might cause performance degradation when the list is loaded due to it having to load more records.