TextAnalysisR Logo

R-CMD-check Project Status: Active License: GPL-3

Text mining and natural language processing workflow for documents (PDF, DOCX, XLSX, CSV, TXT). Includes preprocessing via quanteda, lexical analysis (term frequency-inverse document frequency, log-odds ratios, lexical diversity) via tidytext, topic modeling via stm and BERTopic, semantic similarity and document clustering on transformer embeddings, an interactive Shiny interface with ggplot2 visualization, optional spaCy lemmatization, and local sentence-transformers or web-based (OpenAI, Gemini) model providers for retrieval-augmented generation.

Installation

From R-universe (pre-built binaries for Windows, macOS, and Linux):

install.packages("TextAnalysisR",
  repos = c("https://mshin77.r-universe.dev", "https://cloud.r-project.org"))

Or the development version from GitHub:

install.packages("remotes")
remotes::install_github("mshin77/TextAnalysisR")

First-Time Python Setup (Optional)

Python is optional. Core analyses (topic modeling, keywords, lexical diversity, networks, sentiment lexicons) run in plain R. Use the local package for large datasets.

Several functions (lemmatize_tokens(), generate_embeddings(), cluster_embeddings(), PDF extraction, transformer-based analyses) require Python packages. Run this once after installing TextAnalysisR:

library(TextAnalysisR)
setup_python_env()

This creates a dedicated virtualenv (textanalysisr-env), installs the packages listed in inst/python/requirements.txt (spaCy, pandas, pdfplumber, sentence-transformers, torch, umap-learn, hdbscan, scikit-learn, numba), and downloads the en_core_web_sm spaCy model. Restart R afterward. Check status anytime with check_python_env().

Load the TextAnalysisR Package

library(TextAnalysisR)

Alternatively, Launch and Browse the Shiny App

Access the web app at https://www.textanalysisr.org.

Launch and browse the app on the local computer:

run_app()

Getting Started

See Quick Start for tutorials.

Citation