Yeah I think the Academy has a more complex backstory than maybe they show sometimes... It's perhaps a place where lots of people go, and maybe even non-humans, but where conscription and possible honest recruitment take place too...
That said, think about this. The Empire has slavery, but is it slavery of everyone, or is it slavery of select species? Perhaps under the guise of punishment for "anti-Imperial" behavior, or even anti-Republic (going back to the Clone Wars) actions... Which these things could be largely considered just punishment, and retribution for their actions. It could be established that races, such as Wookiees, were seen as a threat via propaganda, manipulation of the facts, etc., and same with the Lasat, or anyone really.
Meanwhile, there was actual slavery going on prior to TCW, which was considered reprehensible and not even believable to some... Padme's reaction to Anakin's enslavement on Tatooine. So the Empire comes along, and they basically encroach and reign in "Hutt Space" and all these places that were beyond the Republic's realm or sphere of influence. The Empire reigns in these "Disruptive Alien Cultures", like Neimoidians, or Geonosians, or anyone else who started to speak up. Slavery, forced labor, become different things (to people who support the Empire) to rather just punishment... Now of course, some will see this for what it is, but others won't, especially the further removed they are from it.
I've always enjoyed that view of Star Wars, which didn't have the blinders of the movies on... The Empire was maybe expanding what the Republic had. They were exploring wild space, new settlements, making new opportunities for people, regulating things that were unregulated, reigning in the "bad elements" (Piracy, bounty hunting issues, overbearing corporations and conglomerates, etc.), but at the same time you have the obvious BAD things the Empire was doing too, but did everyone always see that? Or did some guys just join up, and have a fairly mundane career?
Regarding Kallus' personal story there, I can take/leave that. I wouldn't have been hurt if they went the opposite way and he's just a horrible Imperial who hates aliens, commits genocide... But I'm totally ok with it going the other way.
Matt, you're gonna hate this, but Force Commander's story was the embodiment of this discussion really. 2 troopers, brothers, serve in the Empire... Time separates them, they learn some about the Empire and their pasts, eventually they're both in the Alliance by the end of the story, realizing the Empire was corrupt (even more than the Old Republic was).