What really needs to be understood about AI LLMs such as Chat GPT and the like is that they *simulate language*. That's what they're doing: *simulating*. They're not thinking or reasoning. You've put in a text block that *you* think is a question or request because you're a human and you can think and reason and ask questions, and the LLM, which is programmed to always give a response no matter what, is simulating a response based on statistics of language texts. We only think there's some meaning to the response because we can think. But the machine isn't thinking and so cannot provide a real response, just a simulation. I think that's really important to understand about how AI LLMs work because it becomes dangerous to believe that the LLM is some sort of wise guru (speaking of which, even Solomon in all his wisdom fell into ruin at the end) just because it's been trained on all the internet information. I'm particularly concerned about Catholic apps like Truthly or even just having an AI chat bot on a Catholic site. No LLM, no matter how much training it has had on Catholic texts, can replace a trained theologian. And considering how easy it is to get an AI to "hallucinate" (aka give a falsehood as a response and insist that it is correct), I wouldn't want that being the way someone is trying to look up Catholic teaching.
I think Bishop Schneider's Catechism AI is a model for how AI searches should be: he restricted it to being trained only on approved Catechisms, so you don't run into the trouble of consulting crazy internet people (like us Pillar readers or worse), and the references to the Catechisms are provided after the AI text answer to your question, so it is easy to check its answer.
I expect it is also not optimized for increasing the length of the user interaction, which is a large part of why AI bots tailor their answers in ways that make them false, and also addictive for some people.
The problem is, you won't necessarily know that an AI bot is set up that way, and even if you dig into it and check, you have no guarantee that it will not be "upgraded" into a liar without announcing the fact.
My concern is that even with references, a user of Catechism AI (a phrase which admittedly makes me gag a little) is likely to run into the Wikipedia problem, ie, the existence of source references makes the user less critical of the claims being made, whether or not they *actually check the sources*. A user could very well receive an AI hallucination in response to some theological/doctrinal question, see that it's referenced to any number of approved catechisms, and go on their merry way without ever checking if the catechisms actually say what the AI is claiming. I'd much prefer a database that lets users easily search the text of these catechisms, without attempting to summarize or synthesize the information contained therein.
I certainly prefer that for my own use, but I can see the helpfulness of putting together an answer that matches the question the user asks over providing references, for other people who might have trouble parsing the original sources.
It won't be perfect, but neither are human teachers. I think the AI hallucinations will be far less of a problem for an AI trained on limited sources, and which does not optimize for user interaction time. I don't think that's a function of NLP as it is of programming to achieve goals that are not simply accuracy. Although, I am perfectly willing to be corrected on that. Not much experience given I stay away from AI and do things old school: read the dictionary one entry at a time...
There's already been a case in the UK where the Judge excoriated both legal teams who relied on AI which promptly hallucinated laws and precedents from whole cloth. It was a civil matter, but you can imagine how embarrassing it is. Any device that plays to human confirmation bias is BAD BAD BAD for everyone. https://substack.com/@hollymathnerd/p-183732867
I've been talking with friends about how long these consumer ai apps will exist (we work at various large tech companies as SWE) and our consensus is not very long. it costs so much for these queries that consumer facing queries are a waste, especially with the heavily discounted services right now. unless we magically generate 20 nuclear power plants and discover sustainable fission these agents will just become another B2B corporate productivity tool. and probably replace all call centers.
The bishops didn't have to be as careful in the past, because they weren't such greedy hypocrites. The USCCB took well over a hundred million dollars to silently participate in the mass trafficking of hundreds of thousands of children throughout this country, many without even a pretense of vetting, and many of whom went to pimps and human traffickers. They repeatedly ignored the many foreign gang members who raped and murdered their way across the US, so much so that we have seen the largest drop in the murder rate in US history in the last year, and they loudly advocated for those people to be here. In the wake of the sex abuse scandal, people aren't willing to naively believe the bishops anymore when they wade into partisan politics, especially when they have an enormous financial stake in the matter.
The USCCB does not traffic children. That is a blatant lie. At some point you will have to answer to God for your idolatry towards politicians and you calumny towards bishops.
I stand by what I said. It is not calumny: it is the truth. The USCCB was silent while hundreds of thousands of unaccompanied minors were moved throughout our country by human traffickers. They often helped resettle these children, and they said nothing about the lack of vetting that they were witnessing first-hand.
The bishops didn't formally join up with the cartels, but they were silent while participating in a corrupt system that saw the time needed to procure a child for sex drop to a mere 15 minutes. Show me the statements under the previous administration condemning cartel control of the border which resulted in an explosion in drug overdose deaths and child prostitution. If you can show me 10% as much protest over child rape as we see over the current issues, then I'll recant what I said. I never saw a single statement from my local bishop. He is too busy lecturing the world about war to care about the war being waged on his own parishioners by the cartels.
Why don't you get off your high horse and visit a border state. A local Walmart that I shop at had a drug dealer's body show up with his head and genitals decapitated: a clear cartel hit. There is a cartel controlled town 30 miles from my house that several police officers have told me is a no-go zone for law enforcement. Land that used to be safe to hunt on was overrun by mules with big backpacks and AK47s, becoming unsafe to visit for US citizens. Over half of women migrants are raped by along the way. A farmer I know grows feed for cattle. A rancher friend of his sent a photo of a rape tree on his property: a tree strewn with the panties of women taken by the cartels as a last act of degradation, right after crossing the border, so the women would know that the US government would do nothing to help them. I witnessed a coyote openly giving a bribe for a couple illegals to get their Real ID driver's license while my daughter got hers. I'm tired of people who don't deal with any of this lawlessness lecturing me about idolatry to politicians. You know nothing about what it is like for people in border states.
Here’s what I’ve been thinking about with regard to AI, as someone who hasn’t used any of the LLMs-
I hear people in my life talking about the creative ways they are learning to use AI tools in their work, figuring out new processes and new ideas for a new technology. They’re all pretty jazzed about it. I like to hear about that kind of creativity. And AI tools have greatly benefitted a physician in my family by cutting down the time he spends documenting patient visits, for example.
But I can’t help thinking that this period of professionals and individuals being creative and entrepreneurial with these tools will be very short, and will pave the way for a much longer period of workers just being slotted in as cogs in a machine, managing processes that have already been engineered. Maybe that is myopic and I don’t have the full picture, but yes, Henry Ford was a genius, nevertheless he created generations of factory workers who didn’t have any agency in their work.
One of my concerns is that people have entirely underestimated the importance of various forms of "drudge work" for getting information into our brains so that we can be creative or troubleshoot things or analyze data. So we'll wind up as mindless cogs because we'll make ourselves unfit for anything else.
I personally hate outlining, because I can typically do it in my head in a much more satisfying way. But the original point of outlining is to help you think things through and figure out what you think. Having an AI do that is ceding your human perogative of thinking.
I totally agree! Just because you put info on paper **doesn't** mean you know anything about that topic. AI as a search-tool can be great; AI as a replacement for actual research, sorting through sources, and coming to conclusions based on study actually strips the virtue from the intellectual life.
I completely agree. As I tell my students, I don't make them do tedious calculations and write lab reports because I expect them to do groundbreaking work (quite the contrary—I know exactly the results I expect from them), I do it because the only way they learn to communicate technical information to other human beings is if they sit down and actually practice it.
What really needs to be understood about AI LLMs such as Chat GPT and the like is that they *simulate language*. That's what they're doing: *simulating*. They're not thinking or reasoning. You've put in a text block that *you* think is a question or request because you're a human and you can think and reason and ask questions, and the LLM, which is programmed to always give a response no matter what, is simulating a response based on statistics of language texts. We only think there's some meaning to the response because we can think. But the machine isn't thinking and so cannot provide a real response, just a simulation. I think that's really important to understand about how AI LLMs work because it becomes dangerous to believe that the LLM is some sort of wise guru (speaking of which, even Solomon in all his wisdom fell into ruin at the end) just because it's been trained on all the internet information. I'm particularly concerned about Catholic apps like Truthly or even just having an AI chat bot on a Catholic site. No LLM, no matter how much training it has had on Catholic texts, can replace a trained theologian. And considering how easy it is to get an AI to "hallucinate" (aka give a falsehood as a response and insist that it is correct), I wouldn't want that being the way someone is trying to look up Catholic teaching.
I think Bishop Schneider's Catechism AI is a model for how AI searches should be: he restricted it to being trained only on approved Catechisms, so you don't run into the trouble of consulting crazy internet people (like us Pillar readers or worse), and the references to the Catechisms are provided after the AI text answer to your question, so it is easy to check its answer.
I expect it is also not optimized for increasing the length of the user interaction, which is a large part of why AI bots tailor their answers in ways that make them false, and also addictive for some people.
The problem is, you won't necessarily know that an AI bot is set up that way, and even if you dig into it and check, you have no guarantee that it will not be "upgraded" into a liar without announcing the fact.
My concern is that even with references, a user of Catechism AI (a phrase which admittedly makes me gag a little) is likely to run into the Wikipedia problem, ie, the existence of source references makes the user less critical of the claims being made, whether or not they *actually check the sources*. A user could very well receive an AI hallucination in response to some theological/doctrinal question, see that it's referenced to any number of approved catechisms, and go on their merry way without ever checking if the catechisms actually say what the AI is claiming. I'd much prefer a database that lets users easily search the text of these catechisms, without attempting to summarize or synthesize the information contained therein.
I certainly prefer that for my own use, but I can see the helpfulness of putting together an answer that matches the question the user asks over providing references, for other people who might have trouble parsing the original sources.
It won't be perfect, but neither are human teachers. I think the AI hallucinations will be far less of a problem for an AI trained on limited sources, and which does not optimize for user interaction time. I don't think that's a function of NLP as it is of programming to achieve goals that are not simply accuracy. Although, I am perfectly willing to be corrected on that. Not much experience given I stay away from AI and do things old school: read the dictionary one entry at a time...
There's already been a case in the UK where the Judge excoriated both legal teams who relied on AI which promptly hallucinated laws and precedents from whole cloth. It was a civil matter, but you can imagine how embarrassing it is. Any device that plays to human confirmation bias is BAD BAD BAD for everyone. https://substack.com/@hollymathnerd/p-183732867
From the General Directory for Catechetics 1997:
“No methodology, no matter how well tested, can dispense with the person of the catechist in every phase of the catechetical process.
The charism given to him by the Spirit, a solid spirituality and transparent witness of life, constitutes the soul of every method.
Only his own human and Christian qualities guarantee a good use of texts and other work instruments.”
I've been talking with friends about how long these consumer ai apps will exist (we work at various large tech companies as SWE) and our consensus is not very long. it costs so much for these queries that consumer facing queries are a waste, especially with the heavily discounted services right now. unless we magically generate 20 nuclear power plants and discover sustainable fission these agents will just become another B2B corporate productivity tool. and probably replace all call centers.
The bishops didn't have to be as careful in the past, because they weren't such greedy hypocrites. The USCCB took well over a hundred million dollars to silently participate in the mass trafficking of hundreds of thousands of children throughout this country, many without even a pretense of vetting, and many of whom went to pimps and human traffickers. They repeatedly ignored the many foreign gang members who raped and murdered their way across the US, so much so that we have seen the largest drop in the murder rate in US history in the last year, and they loudly advocated for those people to be here. In the wake of the sex abuse scandal, people aren't willing to naively believe the bishops anymore when they wade into partisan politics, especially when they have an enormous financial stake in the matter.
https://www.cbsnews.com/news/murders-plummet-crime-trends-2025/
https://x.com/TheKevinDalton/status/2016742453974581666
The USCCB does not traffic children. That is a blatant lie. At some point you will have to answer to God for your idolatry towards politicians and you calumny towards bishops.
I stand by what I said. It is not calumny: it is the truth. The USCCB was silent while hundreds of thousands of unaccompanied minors were moved throughout our country by human traffickers. They often helped resettle these children, and they said nothing about the lack of vetting that they were witnessing first-hand.
The bishops didn't formally join up with the cartels, but they were silent while participating in a corrupt system that saw the time needed to procure a child for sex drop to a mere 15 minutes. Show me the statements under the previous administration condemning cartel control of the border which resulted in an explosion in drug overdose deaths and child prostitution. If you can show me 10% as much protest over child rape as we see over the current issues, then I'll recant what I said. I never saw a single statement from my local bishop. He is too busy lecturing the world about war to care about the war being waged on his own parishioners by the cartels.
https://www.thefp.com/p/inside-americas-fastest-growing-criminal
Why don't you get off your high horse and visit a border state. A local Walmart that I shop at had a drug dealer's body show up with his head and genitals decapitated: a clear cartel hit. There is a cartel controlled town 30 miles from my house that several police officers have told me is a no-go zone for law enforcement. Land that used to be safe to hunt on was overrun by mules with big backpacks and AK47s, becoming unsafe to visit for US citizens. Over half of women migrants are raped by along the way. A farmer I know grows feed for cattle. A rancher friend of his sent a photo of a rape tree on his property: a tree strewn with the panties of women taken by the cartels as a last act of degradation, right after crossing the border, so the women would know that the US government would do nothing to help them. I witnessed a coyote openly giving a bribe for a couple illegals to get their Real ID driver's license while my daughter got hers. I'm tired of people who don't deal with any of this lawlessness lecturing me about idolatry to politicians. You know nothing about what it is like for people in border states.
https://www.youtube.com/watch?v=NUDyY0PSDHo
https://clevelandrapecrisis.org/services/servicios/sexual-violence-latina-hispanic/#_edn3
The Bishops need to be more discerning who they do business with. Having a payroll to meet makes for some bad moral reasoning.
Or you could read The Always War by Haddix where the computers have taken over.
Here’s what I’ve been thinking about with regard to AI, as someone who hasn’t used any of the LLMs-
I hear people in my life talking about the creative ways they are learning to use AI tools in their work, figuring out new processes and new ideas for a new technology. They’re all pretty jazzed about it. I like to hear about that kind of creativity. And AI tools have greatly benefitted a physician in my family by cutting down the time he spends documenting patient visits, for example.
But I can’t help thinking that this period of professionals and individuals being creative and entrepreneurial with these tools will be very short, and will pave the way for a much longer period of workers just being slotted in as cogs in a machine, managing processes that have already been engineered. Maybe that is myopic and I don’t have the full picture, but yes, Henry Ford was a genius, nevertheless he created generations of factory workers who didn’t have any agency in their work.
One of my concerns is that people have entirely underestimated the importance of various forms of "drudge work" for getting information into our brains so that we can be creative or troubleshoot things or analyze data. So we'll wind up as mindless cogs because we'll make ourselves unfit for anything else.
Bingo! Similar with brainstorming ideas, outlining arguments…
Yes!
I personally hate outlining, because I can typically do it in my head in a much more satisfying way. But the original point of outlining is to help you think things through and figure out what you think. Having an AI do that is ceding your human perogative of thinking.
Yep. Thinking will atrophy even more and possibly at a quicker pace.
I totally agree! Just because you put info on paper **doesn't** mean you know anything about that topic. AI as a search-tool can be great; AI as a replacement for actual research, sorting through sources, and coming to conclusions based on study actually strips the virtue from the intellectual life.
I completely agree. As I tell my students, I don't make them do tedious calculations and write lab reports because I expect them to do groundbreaking work (quite the contrary—I know exactly the results I expect from them), I do it because the only way they learn to communicate technical information to other human beings is if they sit down and actually practice it.