From Research Assistants to AI Literacy: Some Reflections So Far

People have asked me dozens of times, “Are you worried ChatGPT will take your job?” One might think I should be concerned. As a Research & Instruction Librarian, I help people find information and scholarship for their research topics. Some might believe that my work could be completely bypassed by generative AI; instead of someone saying to me, “I need help finding scholarly articles on the topic of K-12 literacy curriculum using LGBTQ literature,” they could just ask ChatGPT. This would skip the process of navigating e-resources, such as the hundreds of databases that TC and Columbia subscribe to. It would also avoid looking through and sifting through article results when searching a library catalog. I’ve heard from librarians who work with undergraduates that students see ChatGPT as a way to “hack” the research process.

There are prevailing issues with large language models (LLMs) and generative AI, notably privacy, copyright, inaccurate information, lack of transparency, and bias. These issues are being extensively documented by librarians (Panuncial, 2024). They are measurable and proven (Messeri & Crockett, 2024; Muhlgay et al., 2024). However, there are other issues with ChatGPT and LLMs that are harder to grasp as the technology unfolds. One that I am experiencing firsthand is how ChatGPT prevents users from applying critical thinking to the research process. Asking generative AI for research stops users from learning and understanding the field of scholarship they hope to contribute to. Much is discovered from traditional searching methods in databases—you gain a sense of what scholars are and are not writing about, you notice gaps, and you see the various journals publishing on the topic (Simmons, 2005). This process is iterative; it prompts you to rethink your research questions, explore new paths, or take a different approach. Asking ChatGPT for literature on a topic erases the valuable learning process that occurs when looking for articles systematically. For graduate students, this is detrimental, as we are expected to engage with and contribute to conversations among scholars in our fields. Asking ChatGPT to do your research for you undermines the opportunity to truly become familiar with what our peers are writing about—a task that is fundamental when producing any original research.

There’s no doubt that AI has become a part of research and academia. More instructors are requesting that I address AI in information sessions. In the field of library and information science, articles are being published that discuss AI in research and libraries, and whether librarians are embracing AI tools (Wheatley & Hervieux, 2019). Librarians are under pressure to create evaluative frameworks to guide students in choosing the appropriate tool depending on whether they need it for summarization, feedback, translation, editing, etc.

I don’t tell people they can’t use ChatGPT. Rather, I inform students about the ways ChatGPT can help them, the ways it cannot, and how to evaluate and treat AI as an unreliable source. To use ChatGPT effectively and thoughtfully, you have to understand how it works. It is also important to provide students with information on how these tools are funded and how they negatively impact the environment (Coleman, 2023; Thubron, 2024). This critical pedagogical practice around AI embraces the concept of AI literacy.

Image credit: Digitalpromise.org

AI literacy is the ability to understand, use, and evaluate AI systems and tools. Academic libraries across the country are developing guidelines and programs focused on AI literacy. The topic is still a bit of the wild west, as new tools are being developed with varying degrees of transparency regarding their training data, privacy, and other vague aspects of AI. Here at Gottesman Libraries, we are also beginning to address generative AI in the context of academic research. We will hold a two-part workshop series in October on AI: “AI in Research: Pitfalls and Possibilities” and “Level Up Your Lit Review.” I hope attendees leave these workshops with the understanding that AI tools cannot do their research for them, but they can supplement the process in helpful ways.

Students are typically shy about their use of ChatGPT when I’m providing an information session, so I try to address it at the beginning and stress that ChatGPT's output is a prediction based on data last updated in October 2023 (ChatGPT-4 release notes). Chatbot output is not reliable information. At the end of a recent information session, when I opened the floor for students to share how they use ChatGPT (if they do), one student replied that she found it a useful translation tool for complex academic texts since English is not her first language. While no AI tool is perfect, this was an example of someone finding it helpful.

Many databases are also developing AI chatbot “research assistants” that are integrated into the interface (Enis, 2024). These include Web of Science, Scopus, Primo, and ProQuest, to name a few. The chatbots typically source data only from within the database, not from the internet. For instance, you can ask the Scopus Research Assistant for seminal articles on a topic, and it will provide you with ten articles. This can be helpful if you are just getting started on your research. However, we don’t know how the chatbot chooses those ten articles, which raises real issues regarding transparency and reproducibility. Similarly, if you prompt the Web of Science research assistant with your research question, it will provide citations of articles contained in its collection. A major consideration with these tools is how these proprietary companies will leverage your user data (your prompts, research questions, etc.) to build features that libraries have to pay for.

So, when people ask me whether I’m worried ChatGPT will take my job, I assure them that it has actually added a new dimension to my work. I am teaching students about AI literacy and evaluating these tools against ethical and practical guidelines.

References:

Coleman, J. (2023, December 7). AI’s Climate Impact Goes beyond Its Emissions. Scientific American. Retrieved September 30, 2024, from https://www.scientificamerican.com/article/ais-climate-impact-goes-beyond-its-emissions/

Enis, M. (2024, August 5). AI on the Horizon. Library Journal. Retrieved September 30, 2024, from https://www.libraryjournal.com/story/ai-on-the-horizon

Messeri, L., & Crockett, M. J. (2024). Artificial intelligence and illusions of understanding in scientific research. Nature, 627(8002), 49–58. https://doi.org/10.1038/s41586-024-07146-0

Muhlgay, D., Ram, O., Magar, I., Levine, Y., Ratner, N., Belinkov, Y., Abend, O., Leyton-Brown, K., Shashua, A., & Shoham, Y. (2024). Generating Benchmarks for Factuality Evaluation of Language Models (arXiv:2307.06908). arXiv. https://doi.org/10.48550/arXiv.2307.06908

Panuncial, D. (2024, March 1). The World of AI. American Libraries Magazine. https://americanlibrariesmagazine.org/2024/03/01/the-world-of-ai/

Ruiz, P., Mills, K., Lee, K., Coenraad, M., Fusco, J., Roschelle, J., & Weisgrau, J. (2024). AI Literacy: A Framework to Understand, Evaluate, and Use Emerging Technology. Digital Promise. https://doi.org/10.51388/20.500.12265/218

Simmons, M. H. (2005). Librarians as Disciplinary Discourse Mediators: Using Genre Theory to Move Toward Critical Information Literacy. Portal: Libraries and the Academy, 5(3), 297–311.

Thubron, R. (2024, July 26). OpenAI’s massive operating costs could push it close to bankruptcy within 12 months. TechSpot. https://www.techspot.com/news/103981-openai-massive-running-costs-could-push-close-bankruptcy.html

Wheatley, A., & Hervieux, S. (2019). Artificial intelligence in academic libraries: An environmental scan. Information Services & Use, 39(4), 347–356. https://doi.org/10.3233/ISU-190065