What data may an AI model be trained on?
Reproducing third-party works for training relies on the text and data mining exception. It applies only where access was lawful and the rightholder has not reserved the use.
Training a model means reproducing third-party works. That is permitted through the text and data mining exception, which goes back to the DSM Directive and was implemented in two provisions. Which of them applies turns on who trains and for what purpose.
Two exceptions, two regimes
Reproductions of lawfully accessible works for automated analysis are permitted. Two conditions have to be met together. Access must have been lawful, and what counts is the access rather than whether the content was put online with the rightholder consent. And the rightholder must not have reserved the use. For works available online a reservation is effective only in machine-readable form. The reproductions have to be deleted once they are no longer needed for the analysis.
For scientific research the exception reaches further. A reservation of rights is ineffective, the reproductions may be retained with appropriate security measures and made available to a defined group for joint research and to third parties for quality assurance. In return the group of beneficiaries is drawn narrowly. It covers research organisations, cultural heritage institutions and individual researchers without commercial purposes. Decisive influence by an undertaking together with preferential access to the results excludes the privilege.
What to document
The burden of proof lies with the user, because it is the user who relies on an exception. Checking every single work is impossible with corpora in the millions. A process-level log is not.
- Record per source which reservation signals were present at the time of access
- Evaluate robots.txt, the TDM Reservation Protocol, statements in the source code and HTTP headers, and log that evaluation
- Keep the crawler rule that was applied traceable, so that it can be explained later
- Implement the deletion duty under section 44b(2) sentence 2 UrhG and give reasons for the point in time
- Where a licence is available, prefer it to the exception, because it disposes of the evidential question
- In research collaborations, check whether section 60d UrhG is excluded by commercial influence
How we help
We assign your project to one of the two exceptions, review where the data comes from and set up the documentation so that it answers the evidential question in a dispute. Where the exception does not clearly apply, we settle which licence takes its place.
Your own model, someone else's data?
We check what the training rests on and what you have to document for it.
Get in touchFrequently asked questions
On what basis may we use third-party content for training?
On section 44b UrhG, the text and data mining exception. It permits reproductions of lawfully accessible works for automated analysis. For scientific research section 60d UrhG sits alongside it and reaches further. Both implement the DSM Directive. Whether training large models falls entirely within the notion of text and data mining is disputed and has not been settled by the highest courts.
What does lawfully accessible mean?
What matters is the lawfulness of the access, not of the making available. Content freely available online is covered, as are works to which a licence gives access. Anyone required to establish for every single work whether it was put online with the rightholder's consent could never use the exception in practice.
How effective is a reservation of rights?
For works available online only in machine-readable form, section 44b(3) sentence 2 UrhG. What that requires technically is open. robots.txt, the TDM Reservation Protocol, statements in the source code and HTTP headers are all discussed. The Hamburg Regional Court indicated in the LAION decision that a reservation in natural language might suffice where technology to detect it automatically was available. The Amsterdam District Court did not accept a reservation via robots.txt, without saying what format would have been needed instead.
Who has to prove that no reservation existed?
The user. Because section 44b UrhG is drafted as an exception, the user bears the burden of proof for all its conditions, including the absence of a reservation. The rightholder carries a secondary burden of explaining how and when the reservation was declared. With datasets in the millions this allocation is practically impossible for the user to discharge, which the literature criticises with good reason.
Does something different apply to universities?
Yes, section 60d UrhG goes further in three respects. A reservation of rights is ineffective, the reproductions may be retained with appropriate security measures, and they may be made available to a defined group of people. In return the group of beneficiaries is narrow. A research organisation over which a private undertaking has decisive influence and to which it grants preferential access to the results is excluded from the privilege.