If you have the list of all documents before and after, you let the defendant do the discovery for you

Why is content collection for generative AI so much worse than for search indexes?

Yup, it’s bullshit. Here’s Daniel Ek’s response

I think this is far more common than one would hope. There are many senior developers out there who got their experience in a different time, when test coverage wasn’t important in many businesses. Writing test code is hard and it might be that your teammate simply don’t know how to do it.
If the tests aren’t there at approval time, they will never be there. I think it is perfectly fine to block approval, especially since you all agreed on it.

Solr is a great search engine. It won’t help you with the crawling, but if you manage to get the data into Solr you have a come far.