FT.com [unpaywalled]: “As chief scientist of one of the first companies to use AI in hiring, I built the system that passed you over for that job. The massive employers that were our customers didn’t need to wait for your job application; we in effect applied for you, whether you knew it or not. But when AI decides who gets hired you’ll never know why it wasn’t you. A human being asking you about age, your politics or family planning during the hiring process would be an actionable violation of your civil rights. AI doing the same without your knowledge is just as wrong, but completely hidden from view. Beyond hiring, some employers are toying with systems to automatically fire low-productivity workers. But why fire people if you can “help” them choose to leave? Other companies are developing machine learning-based personality assessments that screen out applicants predicted to agitate for increased wages, support unionisation or demonstrate job hopping tendencies. Loans drive economic mobility in America, even as they’ve been a historically powerful tool for discrimination. I’ve worked on multiple projects to reduce that bias using AI. What I learnt, however, is that even if an algorithm works exactly as intended, it is still solely designed to optimise the financial returns to the lender who paid for it. The loan application process is already impenetrable to most, and now your hopes for home ownership or small business funding are dying in a 50-millisecond computation. AI-driven breakthroughs are occurring in health and biotech every week. In health it is meant to be unbiased, but if you are an outlier then that amazing new diagnostic tool may not diagnose you accurately. How would you know if some unknown company’s proprietary algorithm was trained on patients like you? And if all the doctors in your region are using the same tool, you have nowhere to go for a second opinion. When an AI-powered doctor tells you that you’re just being hysterical, the AI-powered insurance company is unlikely to disagree. In law, the right to a lawyer and judicial review are a constitutional guarantee in the US and an established civil right throughout much of the world. These are the foundations of your civil liberties. When algorithms act as an expert witness, testifying against you but immune to cross examination, these rights are not simply eroded — they cease to exist..”