Written by a human
I spent over a decade in the EdTech world. One of the biggest challenges we all face selling into institutions, whether that be K-12 or higher ed, is complying with the host of information security and other compliance requirements. I’ve got bad news – that’s about to get harder.
AI Regulation
Of course, if you are in EdTech, you’ve been watching AI closely. What you may or may not have been watching as closely is the host of regulations that are about to fall on both vendors and deployers of AI products.
Let’s jump in.
Upcoming Requirements
Let’s start with how regulators are thinking about the issue. A consensus is beginning to emerge as to how many different global regulators are going to converge thinking about AI. This emerging consensus has a couple features:
- A “risk-classification” approach. These laws identify certain AI uses that are simply banned, some AI use cases that are “low risk” and have light or no compliance requirements, and “high-risk” systems. “High-risk” AI systems will have significant compliance obligations
- High-risk systems will be required to have a substantial set of AI governance and compliance documents including:
- A “risk management” system, requiring numerous internal safeguards
- A data policy (and, in the EU, substantial limits as to collection of personal data)
- Impact assessments performed on their technology (in the education context, likely primarily for bias)
Is EdTech AI defined as “high-risk?” Let’s look at the laws. (Note: I’m not a lawyer, definitely not your lawyer, and am not providing legal advice.)
Relevant Regulations
The Colorado “AI Act” (SB24-205) says:
“High risk artificial intelligence system” means any artificial intelligence system that, when deployed, makes or is a substantial factor in making, a consequential decision….”
“Consequential decision” means a decision that has a material legal or similarly significant effect on the provision or denial to any consumer of, or the cost or terms of …education enrollment or an education opportunity.” [Several other high-risk systems are listed]
The EU AI Act includes among the list of high-risk systems (Annex III):
Education and vocational training:
(a)
AI systems intended to be used to determine access or admission or to assign natural persons to educational and vocational training institutions at all levels;
(b)
AI systems intended to be used to evaluate learning outcomes, including when those outcomes are used to steer the learning process of natural persons in educational and vocational training institutions at all levels;
(c)
AI systems intended to be used for the purpose of assessing the appropriate level of education that an individual will receive or will be able to access, in the context of or within educational and vocational training institutions at all levels;
(d)
AI systems intended to be used for monitoring and detecting prohibited behaviour of students during tests in the context of or within educational and vocational training institutions at all levels.
The way I read it, again not providing legal advice, is that quite a few common EdTech applications fail into that bucket, including:
- Anything touching admissions
- Anything touching financial aid decisions
- Grading in some contexts
- Standardized testing in many contexts
Geography
Either of the Colorado or EU law is likely to cover the majority of higher education institutions; there are no limitations on, for example, number of people affected – so in theory if you use AI in admissions on one student located in Colorado, you have violated the law.
Now certainly, K-12 doesn’t have the same geographic scope. That said, be aware that virtually every “blue” and “purple” state are considering substantial regulation of AI systems.
How This Will Go
I’ve been working in AI compliance for several years, which has given me time to see a complete “regulation cycle.” Here is how this went for a law in NYC requiring independent bias auditing of AI systems by employers.
- NYC passes a law requiring employers to conduct a bias audit
- Employer called their vendors. Vendors say the law doesn’t cover them for one legal reason or another
- Employer’s general counsel sees that there is still risk
- Employer call their vendor and say they won’t renew their contract until the vendor gets the audit
- Vendor gets the audit (and pays for it)
So, if you are a edtech AI vendor and show this to your legal counsel, they may tell you that the law doesn’t specifically apply for a certain reason. But I can tell you from experience that almost regardless of the actual wording of the law, if your customers, the institutions’ general counsel, believe that there is even some risk, they are quite likely to require you to comply.
Note also that both Colorado and the EU place compliance obligations on both the developer and the deployer of high-risk AI — so you, the vendor, also may have a legal compliance obligation.
AI compliance is at least as much of a sales and marketing requirement as it is a legal requirement.
What should I do?
If you are an EdTech vendor, you should start preparing the set of AI governance documents that are now required by Colorado and the EU. This is a substantial operation. I liken it to implementing an IT security program like SOC2, but this is harder. Plan for a 2-4 month process.
And yes – Proceptual helps companies implement their suite of AI governance requirements.