# LDE II/21: AI Act

https://www.youtube.com/watch?v=6VwCWxwa3b8

[00:16] Hello, Miss Graver.
[00:23] I already know a lot of your names and I'm happy to see your faces with the names in at the end of September hopefully.
[00:35] I know quite good Mr.
[00:40] Hello, Professor H. great to see you.
[00:48] So, two minutes to go.
[00:53] We are waiting for Miss
[01:01] Good morning.
[01:13] My name is Es
[01:38] So one minute and then we start.
[01:44] I would like to start now.
[01:47] My name is Thomas.
[01:47] I'm a teacher.
[01:51] Uh today talking about mainly the AI act.
[01:56] Um before I start with talking about AI regulation,
[02:01] um I have to say two things.
[02:05] Two things.
[02:05] The first thing still reme please remember what you have to do as a final exam.
[02:12] There are three exams for different um people in this certificate certificate.
[02:22] We have students who have not registered um in the certificate but are min students and he who participate here um for their normal study program.
[02:37] They are not allowed to write an
[02:41] certificate. they have to write a final exam in English at the end um of Jan um July beginning of um July.
[02:52] So please if you are a student for months a normal student don't try to write an an essay you have to write a final exam in English at the end of the term.
[03:08] The next category are Arasmos student students.
[03:13] There are several of them who join this session.
[03:17] So Arasmo students um are not allowed to write a certificate.
[03:24] Um but we have a oral exam for them.
[03:27] There is a quite easy exam not to make life too complicated for you.
[03:30] So it's an oral exam um at the end of the term.
[03:39] And then there's you who registered for
[03:43] The course.
[03:46] You have to write now um final essay.
[03:50] You have chosen topics.
[03:50] I have to say a second thing.
[03:53] I'm a little bit embarrassed um about the process of um telling me what is the topic of your essay because most people made wonderful proposal in time.
[04:03] No, no problem at all.
[04:08] But a few um use the utmost um end of the time and I said I want to see the topic at the end of May at end of months.
[04:25] We said unfortunately um a little bit strange at the the term ends at the 1st of June.
[04:36] So I wonder why there are still these people I think in the last minute of June um of
[04:45] Yesterday, they can send me an email and say, yeah, I'm there.
[04:47] So, we you get an email saying yes, yes, yes, we made the mistake.
[04:52] We didn't say 31 of May, but we said first of everything is fine.
[04:56] But then the story, um, has a follow-up because these people send an email to the LDS list as well.
[04:59] That leads to an error message of the LDA list because the LDA mailing list is only for seen for me and Yonas to write your, um, text.
[05:03] So we had to use hand or hands to stop that error.
[05:07] Then you get a message of an LDA system saying you have sent an error message.
[05:12] Then the story has a follow-up.
[05:15] Um, we got a lot of emails of the people.
[05:47] of Razan at the 4th of June saying oh have you why don't you accept my topic.
[05:52] what I what do I have to do then we tried again to send you an email saying you are registered there's an only an error message of the LDA system which you have used as an error then Again,
[06:13] some of you said, "Oh, why you ever made an error?"
[06:16] And I sent this error to LDS.
[06:19] Oh, so it was a horror what I expected.
[06:25] He seen yesterday it was a nightmare and if you have we now know we have to read I have to read personally 250 essays.
[06:34] So 200 people people are registered and if you have these problems you get oh the feeling of a nightmare.
[06:49] So that's all I want to say um.
[06:52] Before I start and I start with Aidor.
[06:58] You've got several um.
[07:01] [clears throat] text.
[07:04] I've changed my text totally yesterday and sent you a new version.
[07:07] Oh, because work.
[07:14] So, so, so, so, so.
[07:18] Yeah, this because um my old text with 250 satellites doesn't work at all.
[07:25] is um too long too long to send around and I've may produce a shorter text with 50 slides that's much more convenient and what what I like um as well.
[07:39] I integrated the discussion on the digital omnibus.
[07:44] What is a digital omnibus omnibus is not
[07:51] A car, a car system, but it has to do with the European Commission, um, saying we have done a mistake, a big mistake.
[08:01] We have, um, drafted so complicated regulations on AI that AI business is killed, is threatening to be killed as well.
[08:11] So Miss Fonderline said we will start an omnibus procedure for simplification of all digital acts.
[08:22] So there are several omnibus procedures going on.
[08:25] Simplification of the GDPR, simplification of the data act, simplification as well, um, on the AI act.
[08:37] This process I mentioned that here was not stopped at the 7th of May, unfortunately, because the commission made a proposal.
[08:48] This proposal was not accepted.
[08:52] So it was is is still going on in parliament and council what should happened.
[08:58] So you should be aware there is a slight question mark between um for the future what is happening.
[09:09] So see next slide.
[09:16] I said as the 7th of May there was a political agreement on simplification but a formal publication had to be stopped because um council and parliament had a lot of questions.
[09:34] Um you see the the line this um as a first 1st of August um two years ago the AI act entered into force.
[09:49] Um then the 7th of February the prohibitions and the AI
[09:53] Literacy obligations applied.
[09:57] Then it started in the feeling of several enterprises.
[10:05] It's too complicated.
[10:07] We had to do anything and then they had to do um the omnibus procedure.
[10:17] Now the real deadline will un perhaps be the 2nd of August to next year.
[10:27] So um in old slides I've used I've mentioned full application in August.
[10:34] Um this year is that's not true because of the omnivorous procedure.
[10:41] Let us have a look on the background and structure before the AI act um which is by by the way a regulation.
[10:51] So
[10:54] it's binding it in itself.
[10:57] it doesn't need to be implemented.
[10:59] So we have a binding regulation.
[11:02] before the act we had a general principle.
[11:05] we apply the same rules um to AI as compared by uh rules for human conduct is the same.
[11:17] So specific AI rules were really spares and fragmentaryary.
[11:25] Some regulation in Europe hints at transparency and explanability.
[11:32] but that was really rare.
[11:36] What we had is article 22 of the GDPR.
[11:40] very important regulation.
[11:42] um that says um.
[11:46] natural person doesn't um um is protected against having a decision made only by an automated.
[11:57] machine.
[12:01] So um decision which has an impact on human beings shouldn't be in general.
[12:08] This oration was made because the people in France had the feeling they made a mistake because in France immigration processor proceedings were decided by machines and people said you are deciding about the life of a human being only on a based on a machine decision that's not allowed and the French were responsible for having that regulation in the DR as well.
[12:43] So, but in general civil law contract formation and termination was um simply solved um by having human attribution.
[12:54] Um we don't we didn't had
[12:58] Any regulation on machine decisions in civil act in Germany and France and Spain.
[13:06] So the principle of consent, human attention.
[13:21] The idea is um derive a very complicated regulation 500 pages of regulations very difficult to understand.
[13:34] Um, it is aimed at strengthening public trust in AI through control, risk management and legal accountability is promoting voluntary um um high risk requirements.
[13:51] Um um is part of public law.
[13:56] So um it says is very important that the
[13:59] Enterprises accept um special regulations for high risk requirement.
[14:06] What the term high risk means will be defined later on.
[14:12] But there's one element of the AI to distinguish different classes of risk.
[14:19] So that's very important to know in your company which risk classification is valid for our AI system.
[14:31] So it sounds funny but 500 pages regulation supports innovation in the in Europe.
[14:39] That's the reason why Miss Fonder lines notice she might have made a mistake.
[14:47] But she said it we've done it in order to protect safety health on and especially fundamental rights.
[14:56] It's a new way of thinking in public law.
[14:59] Remember your lecture in public law in the second semester.
[15:05] Fundamental rights are directly applicable for the state and state organizations but not for private companies.
[15:14] There's only an indirect influence of fundamental rights for business.
[15:23] This has changed in the AI act because now we have corporation using AI which are fundamentally bound by fundamental rights.
[15:35] That's um for me that was shocking everything I learned as a student is away.
[15:45] So um it creates um um horizontal horizontal meaning covering everything which has to do with problems legal problems of AI risk base.
[15:55] So you still
[16:00] Have to look up the risk base um framework for the use of in the AI industry.
[16:08] Here you see the legislative pass from the first paper white paper to the publication in 2024.
[16:19] Um so it's not very quick but it's not too long four years for having that long paper.
[16:30] When you have a look um in the AI8, you see different chapters.
[16:38] Um you see general provisions that is um a nightmare to understand but you have to check is your tool really AI are you um um working um as a um deployer or a producer of AI.
[16:56] That's chapter one.
[16:59] Chapter two now it starts with this risk
[17:03] classification.
[17:04] We have chapter two is totally forbidden.
[17:09] Totally forbidden that's prohibited AI practices.
[17:11] Then we have a second level.
[17:15] Chapter three that is high risk.
[17:19] That is the biggest danger for you in industry because you don't sell prohibited AI factors.
[17:25] Of course, but you are in danger, danger of um developing a high risk AI system.
[17:35] Let us look later on what it is.
[17:39] Then we have um chapter four for every um AI system even the with limited list a risk they have transparency obligations.
[17:53] Um and it's very interesting to understand what is missing if you're looking where is limited risk or no risk regulation um.
[18:05] there is no one so most companies is 90% have no risk or a minimum risk and they are only bound by this transparency obligations in four.
[18:21] So don't be afraid of the ai act.
[18:26] You should only be afraid if you have a high risk AI system.
[18:32] So and then chapter five very strange thing in the last moment when the European Parliament was discussing the AI8 they notice wow there is a strange thing called um going on there is chat DBT they hadn't been aware of it so they have simply forgotten to deal with it because it was not on the market so it was in total completely new product and they called it general purpose AI
[19:08] model GPAI.
[19:10] and this was regulated in chapter 5 and then of course we have the usual stuff.
[19:18] like governance innovation methods, penalties, um finance and final provisions.
[19:28] So let us go through the act with a scope and core definitions in the first chapter.
[19:37] What is regulated?
[19:37] Who is regulated?
[19:40] Where and for which reason.
[19:46] Now you have a double matrix.
[19:49] You have to solve two questions in before.
[19:53] Who are you and what are you doing?
[19:58] So who are you?
[20:01] There are several roles defined in chapter one.
[20:05] Provider, deployer,
[20:09] Importer, product, manufacturer, authorized representative and affected person and um the user.
[20:21] For imprints the most dangerous roles is provider and deployer.
[20:28] Provider is per definition a person who is developing or has developed um and placed on the market or put into a service under the name or trademark and somebody who's really providing AI systems that is highest responsibility.
[20:54] Compare this person to deployer.
[20:58] He is using a I system under their own authority express purely purely personal non
[21:10] nonprofessional use. The simple user of
[21:14] an AI system of GPI like JTBT
[21:19] is not obliged by anything.
[21:23] So it's only a person using an AI system
[21:27] under their own authority. I've drafted
[21:30] a long um expertise for German
[21:34] universities.
[21:35] Um can universities be simple users,
[21:40] deployers or providers?
[21:44] So for instance if a university says I
[21:48] will say
[21:50] I will give you JTBT
[21:53] as university
[21:55] then you are a deployer because you are
[22:00] offering an AI system under your own
[22:04] authority.
[22:07] But it will be dangerous for
[22:09] universities not to be only a deployer
[22:13] but to be a provider.
[22:16] And that is the case for instance the
[22:19] university of Muna
[22:21] is providing an its own model of JTBt
[22:26] the same for a
[22:29] has a name often
[22:32] and then we're going to use it as
[22:34] student
[22:37] that makes universities to provide us
[22:40] and then they have a lot of duties more
[22:44] than a simple deployer. or user. So it's
[22:48] very dangerous
[22:50] importer is clear and product
[22:53] manufacturer as well. And so the the
[22:57] question main question remains who is a
[23:01] provider who is a deployer.
[23:04] [snorts]
[23:05] So the territorial scope that's another
[23:09] big question
[23:11] does
[23:12] um the AI act
[23:15] um have obligations for American
[23:19] corporations. That's a fear of Vance and
[23:22] Trump who said the Europeans are always
[23:25] trying to kick us in the ass and don't
[23:29] like that.
[23:31] Do you see EU based providers and
[23:34] deployers are immediately covered? Noneu
[23:39] providers are covered when placing AI
[23:42] systems or GPAI model on the EU market
[23:47] or offer services in the EU.
[23:52] So noneu providers and deployers are
[23:55] also covered when the output
[23:59] um um has an impact on the European
[24:05] market
[24:06] that's is very difficult to determine
[24:10] especially Xant
[24:12] um are in output as an output intended
[24:16] or expected to be used in the EU.
[24:20] So in general you could say all big
[24:24] providers
[24:26] of um
[24:28] um GBT services of LLM like glo like um
[24:34] Germany
[24:35] um they are all bound by the AO AI8
[24:43] that was a complicated story to define
[24:47] AI system.
[24:49] So there was a lot of discussion to
[24:52] discuss that sentence in article three
[24:55] subsection two as a machinebased system
[25:00] designed to operate with varying levels
[25:03] of autonomy. It may exhibit
[25:07] an ad adaptiveness update deployment and
[25:12] it has an input and an output
[25:15] and the output may influence physical or
[25:19] virtual environment. Some people always
[25:22] said this is so broad that even a simple
[25:27] software as such can be an AI system. So
[25:32] we have really problems in detail to
[25:37] solve that.
[25:40] We have of course to distinguish models
[25:43] and systems
[25:45] for instance especially in the case of
[25:49] GPAI like JTBT.
[25:52] We have an IM DPI model.
[25:57] Um and the system for instance JTBT
[26:02] is in general as um a system
[26:06] but it may be based on a model
[26:10] and the underlying
[26:13] structure of staticity.
[26:19] There are several exclusions
[26:23] um people are taking out military
[26:26] defense, national security, that's
[26:29] always the same thing for universities.
[26:32] That's very interesting. Scientific
[26:35] research and development is taken out.
[26:38] But if a [clears throat] university
[26:41] plans to earn money afterwards,
[26:45] you are part of the AI act. So still
[26:50] only the pure research as such is has an
[26:54] exemption of the AI act.
[26:58] So if you using AI purely personal
[27:02] non-professional, you are totally out.
[27:06] So a private user of JTV is not a
[27:09] problem. And we have a nice story on
[27:14] opensource AI is um has limited
[27:20] um exclusions. There was a nice trick of
[27:23] um meta. Facebook said we have um
[27:27] created um AI system called meta. we can
[27:33] um you use it um as part open source you
[27:37] can use it for free um so we are not
[27:41] part of this AI act no no no simply to
[27:47] say you are open source doesn't make
[27:50] open source to open source because
[27:52] there's a long definition what is open
[27:54] source which needs to be followed simply
[27:59] open source cannot be found by Facebook
[28:03] in Facebook.
[28:05] So
[28:10] that
[28:11] is a role and the definition. Now I'm
[28:15] coming [clears throat] to the second
[28:16] question. You have to define the
[28:19] riskbased level. So
[28:24] um totally prohibited
[28:27] high risk, minimal risk and no risk at
[28:31] all. Um
[28:35] so these are the levels
[28:38] unacceptable risk according to article
[28:41] five that's prohibited practices. High
[28:46] risk is permitted but heavily regulated.
[28:51] I think overregulated.
[28:53] Um
[28:55] the minimum the medium risk um general
[29:00] risk which is remaining has article 50
[29:05] transparency obligations
[29:08] and if you have no risk at all you
[29:12] almost taken out of the AI
[29:16] and you have five separate regulations
[29:20] for GPA. I am.
[29:24] Is there any question? I see you buying
[29:27] coffee. Coffee coffee is eating.
[29:32] No, I don't see any question.
[29:36] There's somebody who has don't have a
[29:40] microphone.
[29:41] There's no sector specific regulation
[29:45] for military AI M.
[29:48] And um but I have to say um there might
[29:52] be one is a plan of the commission to
[29:54] have a sector specific guideline for
[29:58] military AI.
[30:02] So
[30:05] now let us go further on.
[30:11] Let us use the model of different levels
[30:14] of risk. First category unacceptable
[30:18] risk.
[30:20] Um most um things are not interesting
[30:24] harmful manipulation
[30:27] harmful exploitation social scoring like
[30:30] in China
[30:32] individual criminal offense assessment
[30:35] that's only for state organizations.
[30:39] What will be new definitely coming is
[30:42] this famous um man um regulation. You
[30:48] know Christoff Olman in Germany
[30:52] um very nice but crazy comedian
[30:56] um and his former wife complained that
[31:00] he took a photo and combined it with AI.
[31:07] That's called nudie fire. So and that
[31:11] will be part of the omnibus proceeding
[31:14] definitely become to prohibited to to
[31:18] prohibit nud fire AI use um as part of
[31:23] prohibited risk.
[31:31] Um
[31:33] so as I was
[31:37] So what is high risk? That's article six
[31:42] and annex one and three. Um high risk um
[31:47] is always a high risk when an AI system
[31:51] is a product covered by NX1. That's
[31:55] really rare. Um
[31:58] more dangerous is anx3. is a generally
[32:03] high risk where this I system falls
[32:07] within anx3 and there's a list for him
[32:12] and I would only like to mention two
[32:15] topics mentioned there there I have to
[32:19] change
[32:22] sorry
[32:25] um
[32:27] there's education
[32:29] that's important
[32:32] universities
[32:34] that and employment
[32:38] that is HR
[32:40] they are two broad topics which are
[32:44] always part of anx3.
[32:48] So the universities are now really
[32:51] looking forward to find a good solution
[32:54] in order to behave well in this special
[32:58] class.
[33:00] You can say as a company I'm although
[33:04] I'm working in the area of education and
[33:07] employment I'm not high risk
[33:12] and that um declaration
[33:15] um has been justified by saying we don't
[33:20] pose on significant risk of harm to
[33:24] health safety and fundamental rights. So
[33:29] you can claim um exemption
[33:33] but this exemption must be assessed and
[33:37] documented by the provider. So there's a
[33:40] lot of explanation
[33:43] um you have to justify in front of the
[33:46] commission um if you think we don't have
[33:50] to do with significant risk especially
[33:55] to do that then significant risk to
[34:00] fundamental rights that's very
[34:02] complicated to justify
[34:07] under the omnibus timeline time. The old
[34:10] one was um this year. The new one will
[34:15] be, I hope so,
[34:18] um the 2nd of December, 2027. So,
[34:22] there's still time left. But don't dare
[34:26] to take your feet and say, "Oh, there's
[34:29] a lot of time for me to classify my AI
[34:35] tools." because time is flying now. So
[34:39] you have to really be quick.
[34:45] So
[34:48] any AI system, it doesn't matter how far
[34:54] you
[34:56] classified within this model of risk. Um
[35:00] any ITLE um is under transparency
[35:04] obligation. So humans must be informed
[35:08] when interactive with an AI system. Um
[35:12] providers of generative AI must ensure
[35:17] that their content is identifiable in a
[35:21] machine readable way.
[35:24] That's um is timeline
[35:28] 3rd 2nd of December next year. But then
[35:33] every provider
[35:36] even of share TBT
[35:39] um has to identify a AI generated
[35:44] content.
[35:46] There's a prohibition of deep fakes.
[35:49] Nobody knows what deep fakes are. We
[35:51] have already it's consulted under
[35:53] article 25 of the DSM.
[35:58] Defects should be taken care of
[36:02] and emotional recognition and biometric
[36:06] categorization system
[36:08] has been regulated separately.
[36:14] If you are
[36:17] no or minimal risk model
[36:21] and system
[36:24] you are bound by the AI act but you fall
[36:28] outside high risk or prohibit prohibited
[36:31] categories
[36:33] as for in the case um in of AI enabled
[36:38] video games spam filtering
[36:42] basic recommendation tool
[36:45] without high risk use.
[36:50] So
[36:52] the next chapter
[36:54] obligations
[36:56] for the bad case you are and providing a
[37:01] high risk AI
[37:04] then it becoming very expensive and
[37:07] really hard because you have to build up
[37:11] a risk management system. You have to
[37:14] provide data and data governance.
[37:18] Um you have to provide an extensive
[37:23] technical documentation.
[37:26] Um you have to um
[37:30] make it transparent who is logged in and
[37:34] who is gets records. Um
[37:38] you have to take care for transparency
[37:43] and you have to provide at least at the
[37:46] at one point a human oversight.
[37:50] So somebody should take care taken care
[37:53] and have the last um decision upon a
[37:57] certain use of AI
[38:00] and of course cyber security is as well
[38:04] a good topic. Here you see a risk
[38:07] management that
[38:10] obliges you to identify known and
[38:14] foreseeable risk. Um to estimate and
[38:18] evaluate risk to health safety and
[38:21] fundamental rights. You see it's always
[38:23] the same and to mitigate risk and to
[38:28] test AI before placing it on the market.
[38:34] Data governance
[38:36] means training, validation, testing of
[38:40] data sets must be relevant, sufficiently
[38:44] representative and free of errors and
[38:48] com
[38:50] um complete.
[38:52] You have to provide a technical
[38:54] documentation before you um allow the AI
[38:58] to go on market
[39:00] and it must be kept for 10 years.
[39:04] And the instruction for use of this
[39:07] documentation must be clear, complete,
[39:10] correct and comprehensible for
[39:13] deployers.
[39:17] You have to provide a effective human
[39:20] oversight
[39:22] and operators need to understand the
[39:25] capabilities and limitations of the
[39:28] system. The oversight must allow
[39:31] monitoring, detection of and un
[39:34] anomalies
[39:37] and interruption.
[39:40] And last cyber security, cyber security,
[39:44] cyber security.
[39:46] So providers have to
[39:51] take care for these obligations
[39:55] as well as the deployers.
[39:58] they have a reduced amount of duties
[40:03] um when it comes to documentation.
[40:07] And the last chapter five is a GPAI
[40:12] andic risk.
[40:18] So um that was entered into the AI um at
[40:23] the last moment. Providers of GPAI
[40:27] models must keep a technical limitation.
[40:30] They must make information available to
[40:33] providers. They must put in place a
[40:36] policy to comply with union copyright
[40:39] law. I'll come to that. They must
[40:41] publish a sufficiently detailed summary
[40:45] of training consent.
[40:47] Um that was very long. And if you have a
[40:51] systemic risk GPI AI, you are
[40:55] additionally bound by article 51 and 52.
[41:01] Systemic risk means you are building up
[41:05] a very big um GBT
[41:09] LM that exceeds 10 to 25 flops. So it's
[41:15] a really big trip and very quick. Um
[41:19] these um are kind of high risk um LLMs
[41:26] and um the commission has already um
[41:29] published the guidelines how to define
[41:32] it and what um DPI AI providers are
[41:38] supposed to be doing.
[41:41] So
[41:44] then there is a code of practice for
[41:47] seen a voluntary road to comply with all
[41:51] this transparency copyright safety. I
[41:54] will provide you with an additional
[41:57] slide on the question what is regulated
[42:00] in this code of practice.
[42:05] So the digital omni was at the end
[42:08] very important.
[42:13] Um
[42:14] so
[42:16] high risk obligations are ticking out
[42:19] from the term August
[42:22] 2026.
[42:24] Um is aimed at simplification and
[42:28] postponement. Postponement not a repeal.
[42:32] It was the idea of having a sandbox
[42:36] world in Europe. So that at the
[42:39] beginning new companies who want to
[42:42] build up an AI structure as a startup
[42:46] should be free from any regulation. But
[42:49] that was not a view of the commission.
[42:53] So
[42:55] um
[42:58] so so on we are instead we have a world
[43:04] where the AI literacy obligation
[43:09] um is still there. So you have to inform
[43:13] all your employees who are working with
[43:15] AI on the system the internal structures
[43:21] to use AI. You have to build up a kind
[43:25] of AI compliance road map.
[43:29] I'm really happy that most people of you
[43:32] are taking part in the additional
[43:34] program um the ITM's offering for the
[43:38] question of AI compliance.
[43:46] Open questions. Um
[43:49] we have some open questions. Um is that
[43:54] really only public law? That sounds like
[43:58] public over regulatory law. But what is
[44:04] the influ influence of the AI8 in civil
[44:08] law? For instance, we have section 823
[44:16] of our civil act. It says that several
[44:20] that statutes um from public law may
[44:25] even be part of tor law. So
[44:29] how far does the AI8
[44:34] has an influence on section 823
[44:39] or
[44:41] what will happen and how will courts
[44:44] handle documentation defects in
[44:46] liability cases? Is it a question of
[44:51] liability under contract law so that the
[44:54] AI act and the AI standards in the AI
[44:58] act influence the liability in contract
[45:02] law?
[45:04] Um what is happening with extrator
[45:09] territorial output use? can we apply and
[45:14] how far can we apply the AI in a case of
[45:19] extr territoriality.
[45:22] So let ask some questions which remain.
[45:28] Um I have tried
[45:31] to write two case studies for you to at
[45:34] home. You get the uh amounted um slides
[45:39] afterwards
[45:41] but I prepared two case studies in order
[45:47] to show you how the system works.
[45:51] Scenario
[45:53] one, a company uses AI tools to rank CVs
[45:59] and predict cultural fit [snorts]
[46:02] and s suggest interview shortness.
[46:06] The vendor markets the product as a
[46:08] productivity tool. The employer controls
[46:12] criteria and deployment.
[46:16] model output is not final but managerial
[46:19] rarely deviates from the ranking.
[46:23] If you see the AI
[46:27] act, this is a system which is likely to
[46:32] be a high risk system under anx3
[46:37] employment.
[46:38] It says employment worker manage access
[46:42] to a self-employment is part of this NX3
[46:47] high risk. So the provider has to make
[46:51] risk assessment, data governance,
[46:54] technical documentation and human
[46:57] oversight. The deployer
[47:00] that is the
[47:03] commercial user
[47:06] um of the system.
[47:09] He has to take care
[47:12] um for the instructions that he uses
[47:14] according to instructions monitoring.
[47:18] He has to provide a competent home or
[47:21] human oversight
[47:23] and that's all.
[47:27] So
[47:29] second case
[47:31] you can try to solve the issue um of
[47:36] credit scoring. A bank deploys an AI
[47:39] model to estimate
[47:42] default risk and automatically proposes
[47:46] loan limits. The inputs include
[47:49] transaction data and employment status.
[47:54] The customer receives a short rejection
[47:58] notice. The scoring logic remains
[48:01] bizarre. Nobody knows it.
[48:05] When you take the AI8
[48:08] is again an X3 access to essential
[48:12] private services
[48:15] for worthiness as I mentioned in Alex 3
[48:18] high risk.
[48:20] So the GDPI is also relevant and because
[48:25] now we have a case where personal data
[48:27] are stored. So if you take article 22 in
[48:31] the case of credit scoring you have to
[48:34] provide a human who is deciding about
[48:37] this credit score and in addition you
[48:42] have high risk obligations risk control
[48:46] and so on. So
[48:52] I've chosen a third case study. Um
[48:57] hospital medical data purchases a
[49:00] diagnostic imaging system that flags
[49:05] suspicious lesions. The system is
[49:08] embedded in medical device system and
[49:12] influences clinical prioritization.
[49:16] The provider updates the model
[49:19] periodically after market pre-launch.
[49:23] If you only take a view of the AU, you
[49:26] see it's high risk because it's a soma
[49:31] component of a product subject to EU
[49:35] harmonization rule that is um the ANX1
[49:40] that is always high risk. So you don't
[49:43] need to decide because there are medical
[49:46] products are part of a separate
[49:48] regulation um of the EU
[49:51] you are in
[49:53] but and provider must therefore
[49:57] provide quality management post market
[50:00] surveillance and must take care for
[50:04] change management when the model updates
[50:08] has been altered.
[50:11] The digital omnibus in that case um is
[50:14] planning to open to liberize um um the
[50:18] word for medical AI products but let us
[50:22] see what is happening we don't have um
[50:27] final end of the discussion
[50:30] so
[50:33] last slides
[50:35] are we alone are we alone in the world
[50:39] no others are working as well. We have
[50:43] of course um we are proud to say we have
[50:46] the longest regulation
[50:48] most complicated regulation on AI the AI
[50:52] act. United States doesn't have one um
[50:58] they are planning it on the basis of
[51:00] state laws and we come to that in a
[51:03] second. We have already plans from
[51:07] United Kingdom.
[51:09] Um we have uh guidelines from China and
[51:14] we have regulations from South Korea.
[51:17] That are the seven countries which have
[51:22] um the longest relation of that and I
[51:25] try to show them um in blocks. We have a
[51:31] product safety model
[51:34] meaning
[51:35] the AU act is a public law regulation of
[51:40] everything internally uh in the
[51:43] production of AI. We have an enforcement
[51:47] model is based on state laws. No federal
[51:53] um AI act. It has to do with um Trump
[51:56] who is totally dissatisfied
[52:00] afraid of um regulating AI business too
[52:05] much. So no federal AI act but certain
[52:09] states within the
[52:12] US
[52:14] um made regulations for consumer
[52:16] protection for civil rights and sector
[52:20] law regulations.
[52:23] The UK and Japan have a very nice innov
[52:27] po pro innovation stratey say that we
[52:31] don't want to kill AI. So we open our
[52:34] laws to allow a startup um in the area
[52:40] of AI and we wait until we know what is
[52:44] happening in the future.
[52:46] The sovereignity model that's China.
[52:50] China is saying if you earn money it's
[52:53] okay do whatever you want to do but if
[52:58] AI is a danger for the state then you're
[53:02] out so don't do that
[53:07] so
[53:10] that's all for that and I prepared a
[53:14] slacking slide which of course you will
[53:17] get after this lecture
[53:20] because I wanted to show you.
[53:25] So
[53:29] that mission is not finished with having
[53:33] a discussion discussing an AI.
[53:37] They are preparing several papers. Um I
[53:41] call them sub subtext papers. They are
[53:46] not legal text.
[53:49] But they are very important for
[53:51] companies in order to show
[53:55] to the European Commission I am okay. I
[53:58] don't care with me. I'm fine.
[54:02] So they um made a code of practice and
[54:07] they made commission guidelines. The
[54:10] last guideline was published last week.
[54:14] Um I will show that to you.
[54:20] So [snorts]
[54:22] the AI AI creates binding duties and the
[54:27] guidelines explain how the commission
[54:30] understands these duties
[54:33] and the general purpose AI code of
[54:36] practice translates the statutory
[54:39] obligations for LLMs like JTBT into
[54:45] operational measures.
[54:48] So then at the end the practical
[54:51] question should no longer be what does
[54:54] the X say but which interpretive
[54:59] layer applies. You have to see all the
[55:03] papers.
[55:06] The code of practice is um was developed
[55:10] by independent experts. So a long
[55:13] process of I think three years
[55:17] um is advised to providers of uh JTBt
[55:21] and co. So LLMs
[55:24] um
[55:26] and the commission and AI belt treat it
[55:30] as an adequate voluntary to tool to show
[55:34] demonstr demonstrate compliance.
[55:38] These are the three blocks which are
[55:41] regulated. [clears throat] Transparency,
[55:44] copyright, very important issue and
[55:47] safety and security.
[55:51] Transparency.
[55:55] If a mission wants to explain what they
[55:57] meant with transparency.
[56:01] Um,
[56:03] so you have to maintain a model
[56:05] documentation with the intended task and
[56:09] capabilities and limitations of your GBT
[56:13] of your
[56:18] one second. I have to go to the door.
[56:37] Thank you.
[56:44] So,
[56:47] so you have to provide a model card and
[56:50] technical file as a structured
[56:53] documentation form um so that um
[56:57] downstream providers can assess their
[57:00] own duties.
[57:02] Um that's very important to understand
[57:06] how you provide
[57:08] um transparency
[57:11] copyright of course we have spoken in
[57:14] winter term on copyright problems of AI
[57:19] and you may remember the famous lion
[57:21] case of the district court of um Hamburg
[57:26] and and very correctly the district
[57:29] court said why Are you complaining as a
[57:33] photographer about your photos being
[57:37] stored in an AI system?
[57:41] Because that's mentioned in the AI act
[57:45] as data mining. So I don't understand
[57:49] GMA and the others why they are so
[57:52] heavily fighting for the opposite view
[57:56] is already regulated.
[58:01] So but in the code of practice providers
[58:06] should document how they identify and
[58:10] respect um copyright.
[58:14] Um the training data governance should
[58:17] become a compliance function. That means
[58:21] the records, policies and provider
[58:24] accountability
[58:27] should taken care of rights and and copy
[58:31] especially copyright
[58:35] and
[58:37] security.
[58:39] As always, what does security means?
[58:41] Because in the AI act, AI security is
[58:45] only mentioned as a very broad goal.
[58:49] Um to see you what should be done you
[58:53] have to take read the chapter [snorts]
[58:56] um in the u code of practice
[59:01] there is certain standards
[59:05] then we have the guidelines and they are
[59:08] new they are published um last week
[59:14] um as a draft so they are not final The
[59:19] commission wants to um see how the
[59:22] omnibus procedure is going on and
[59:25] therefore they're waiting um until this
[59:29] omnibus procedure is finished and then
[59:32] they publish a final read final draft.
[59:36] But in the graph um is already mentioned
[59:41] the definition of an AI system.
[59:46] Then
[59:48] as well prohibited practices. what is
[59:52] strongly prohibited including new fire
[59:55] and the other things
[01:00:01] um
[01:00:03] and
[01:00:08] as as well as a code of practice
[01:00:12] commission guidelines are non-binding
[01:00:14] but you have to take care for the courts
[01:00:17] and authorities they still treat them as
[01:00:22] persuasive.
[01:00:24] So you should have a look if you are
[01:00:27] advising a company.
[01:00:31] So that's all for that. But I am not
[01:00:35] finished.
[01:00:37] I have a third slide for you
[01:00:42] and I will use that
[01:00:45] is in a second
[01:00:49] and that has to do with
[01:00:52] a very important topic.
[01:00:55] Who is liable
[01:00:58] for AI?
[01:01:01] AI liability because at the end of
[01:01:05] telling you a lot about mistakes which
[01:01:10] may may be made you have to be aware who
[01:01:14] is responsible for what mistake
[01:01:20] we have um
[01:01:23] starting point in private law I'm not
[01:01:26] speaking about public law liability I'm
[01:01:29] speaking about private All we have um
[01:01:33] two um three um blocks. One is contract,
[01:01:40] one is port based to classical court to
[01:01:45] law and we have split product liability
[01:01:52] contract is quite easy. You know when a
[01:01:55] AI system has a defective performance
[01:02:00] if it's sold by by together with
[01:02:03] inadequate instructions
[01:02:05] is that's a failure to update that is
[01:02:09] always a problem of contract liability
[01:02:14] that particularly relevant in B2B supply
[01:02:17] chain cases.
[01:02:19] Um the problem still remains loss
[01:02:24] allocation that's very difficult to find
[01:02:27] a judge to decide that as part of
[01:02:30] contract liability
[01:02:33] then we have the in the middle fault
[01:02:37] base to um you know as German lawyer
[01:02:42] section 823
[01:02:46] um that is still the core principle
[01:02:49] In the AI world, you have to prove a
[01:02:53] duty,
[01:02:55] then breach of duty, then causation,
[01:03:00] then a mistake, and then a violation of
[01:03:04] protected interest.
[01:03:07] The main difficulty is um it's so
[01:03:11] different to prove it um for instance
[01:03:14] the causation of certain errors and
[01:03:17] certain damages were caused by AI is
[01:03:22] always a nightmare
[01:03:25] and next
[01:03:27] and that will become more and more
[01:03:29] important a strict product liability
[01:03:33] under the directive of 2024 for because
[01:03:37] it expressly covers software and AI
[01:03:41] enabled products that are product
[01:03:44] liability products and that operates
[01:03:48] independently of fault but still you
[01:03:51] have problems improving
[01:03:54] defects and causation.
[01:04:00] That is the reason why I think AI is a
[01:04:03] pressure
[01:04:05] on traditional liability doctrines and
[01:04:08] nobody knows what to do. We had a
[01:04:10] proposal on a separate AI liability
[01:04:15] directive
[01:04:17] but that was
[01:04:20] I don't agree with everything in which
[01:04:23] it was regulated. It was not a very good
[01:04:26] proposal. So, Miss Fonder decided to
[01:04:30] skip the proposal in the last minute of
[01:04:34] um
[01:04:35] their his her power. So, she stopped it
[01:04:41] and waited till the next election in
[01:04:44] parliament.
[01:04:47] So now we have the product liability
[01:04:50] directive.
[01:04:53] The scope software is included
[01:04:57] especially included
[01:04:59] digital manufacturing files are also
[01:05:03] products and even AI enabled goods and
[01:05:08] standalone software
[01:05:10] uh within the scope of this directive.
[01:05:14] This is in the process
[01:05:17] of being implemented in Germany. So
[01:05:22] during the next weeks, our parliament is
[01:05:25] deciding
[01:05:27] upon a new product liability regulation
[01:05:30] as part of German law. And then you have
[01:05:33] to do and earn a lot of money. Um with
[01:05:38] all these new product liability rules,
[01:05:42] they are very important um for
[01:05:46] manufacturers and quasi manufacturers.
[01:05:50] importers and even distributors and
[01:05:54] certain platform actors are already
[01:05:58] regulated in that liability directive.
[01:06:03] Damage is really broad
[01:06:07] uh death and personal injury. So if you
[01:06:11] use AI in a hospital, you have an
[01:06:16] increased danger of being sued for
[01:06:20] product liability.
[01:06:23] Poverty damage are also part of this
[01:06:26] product liability. And in the past they
[01:06:29] had a 500 euro threshold that was
[01:06:33] limited. So you don't have any limit.
[01:06:39] So data loss or corruption are still
[01:06:42] outside. They are not part of this
[01:06:46] directive.
[01:06:50] What is a defect of an AI product? Um
[01:06:56] traditional score call say a product is
[01:06:59] defective if it's um does not provide
[01:07:02] the safety that a person is expecting.
[01:07:07] The digital um extension says um you are
[01:07:12] also responsible for software updates,
[01:07:15] upgrades, connectivity
[01:07:18] and foreseeable interaction with other
[01:07:20] systems and AI specific is regulated
[01:07:26] with a self learning capability and
[01:07:30] postmarket evaluation matter matter
[01:07:33] because the relevant risk may
[01:07:36] materialize after first deployment
[01:07:40] and you are even liable for mistakes in
[01:07:45] the area of cyber security.
[01:07:51] Um you have to there's one good thing
[01:07:55] you have still a high level of evidence.
[01:08:00] So we have to prove that a damage
[01:08:05] causes that a mistake causes a damage
[01:08:10] but there's a presumption
[01:08:14] that certain mistakes have caused
[01:08:18] typically this damage the defectiveness
[01:08:22] causation or both may be presumed where
[01:08:26] the technical or scientific complexity
[01:08:30] grace extends excessive difficulties of
[01:08:34] proof.
[01:08:39] So you see the German implementation
[01:08:43] um is is now in the parliament. It will
[01:08:46] be discussed now now. So please be aware
[01:08:50] what is happening in parliament.
[01:08:56] What does it mean for
[01:08:59] laymans and for manufacturers?
[01:09:03] The person viewing
[01:09:06] should focus on access to evidence, log
[01:09:09] in architecture, update chains and
[01:09:13] retains control over the system.
[01:09:16] Um, he should frame AI harms either as
[01:09:21] product defects or as breaches of
[01:09:24] specific duty under general to law
[01:09:29] and
[01:09:31] as should separate defect from damage.
[01:09:38] For the manufacturer,
[01:09:40] he should treat software governance and
[01:09:43] full vulnerability
[01:09:45] and update documentation as liability
[01:09:48] questions.
[01:09:50] He should preserve technical records to
[01:09:54] show in five years for instance that he
[01:09:57] has done everything he could do um for
[01:10:03] um structuring his software
[01:10:06] and he should coordinate AIA compliance
[01:10:11] with civil liability. Don't
[01:10:15] you don't should don't use two different
[01:10:19] departments in a big company. One is for
[01:10:22] AI act the other is for civil liability
[01:10:26] that there is a strong interlink between
[01:10:30] both sides of the metal.
[01:10:35] So this is all for today.
[01:10:40] I hope you have liked it.
[01:10:44] Is there any question left which I
[01:10:46] haven't answered?
[01:10:52] So, so, so thank you.
[01:10:55] We will see each other next Tuesday.
[01:10:58] Then I'm back in Germany and not in bad
[01:11:02] hotel. No, not in bad hotel. Nice hotel
[01:11:05] in TM.
[01:11:07] So, see you next Tuesday.
[01:11:11] Good luck. Nice week.
[01:11:14] >> Thank you.
[01:11:14] >> Thank you very much.
[01:11:16] >> Thank you. Bye.
[01:11:17] >> Have a good week.
[01:11:18] >> Thank you very much.
[01:11:19] >> Byebye.