---
title: "Annotating open-ended survey responses end to end"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Annotating open-ended survey responses end to end}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
fixture_dir <- "annotation-workflow"
recording <- nzchar(Sys.getenv("FOUNDRY_RECORD_DOCS"))
have_fixtures <- dir.exists(fixture_dir) && length(list.files(fixture_dir)) > 0
run_api <- requireNamespace("httptest2", quietly = TRUE) &&
  (recording || have_fixtures)

# Attach foundryR before start_vignette(): httptest2 only sources the package's
# inst/httptest2/start-vignette.R (which sets replay placeholders) from attached
# packages.
library(foundryR)

if (run_api) {
  httptest2::start_vignette(fixture_dir)
}

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  eval = run_api
)
```

Open-ended survey responses are valuable because respondents can say what the
researcher did not anticipate. They are also expensive to code by hand. This
vignette shows a foundryR workflow that keeps each model step visible in a
tibble: extract structured labels, run the same prompt at scale with Batch,
embed text for similarity work, and check that generated findings are grounded
in the source responses.

## Example data

```{r data, eval = TRUE}
library(foundryR)
library(dplyr)

responses <- tibble::tibble(
  respondent_id = 1:6,
  response = c(
    "The lectures were clear, but the weekly quizzes felt rushed.",
    "I liked the examples in R. More office hours would help.",
    "The project made the material practical.",
    "I struggled because the instructions changed late.",
    "The instructor explained regression well.",
    "The course needed more examples before the final exam."
  )
)
```

## Extract structured annotations

Start with a JSON Schema. Keep the schema small enough that a human reviewer can
understand it.

```{r schema, eval = TRUE}
annotation_schema <- list(
  type = "object",
  properties = list(
    sentiment = list(type = "string", enum = c("positive", "negative", "mixed")),
    primary_theme = list(
      type = "string",
      enum = c("instruction", "assessment", "support", "materials")
    ),
    needs_followup = list(type = "boolean"),
    short_summary = list(type = "string")
  ),
  required = c(
    "sentiment",
    "primary_theme",
    "needs_followup",
    "short_summary"
  ),
  additionalProperties = FALSE
)
```

`foundry_extract()` uses strict JSON Schema mode by default for supported
models. The result is one row per response, with schema fields as columns.

```{r extract}
annotations <- foundry_extract(
  responses$response,
  schema = annotation_schema,
  instructions = paste(
    "Code each survey response for a course evaluation.",
    "Use the respondent's words. Do not infer facts that are not stated."
  )
)

coded <- bind_cols(responses, annotations)
coded
```

## Move the same job to Azure Batch

Interactive extraction is useful while designing the schema. For larger jobs,
write a JSONL request file and submit it to Azure's Batch API. The submission
below requires Azure credentials and is not run during rendering. Its temporary
request file is removed after upload.

```{r batch, eval = FALSE}
jsonl <- tempfile(fileext = ".jsonl")

foundry_batch_requests(
  responses,
  input = "response",
  path = jsonl,
  model = "gpt-5-nano",
  endpoint = "/v1/responses",
  body = list(
    instructions = paste(
      "Code each survey response using the supplied schema.",
      "Return only JSON that conforms to the schema."
    ),
    text = list(
      format = list(
        type = "json_schema",
        name = "CourseEvaluationAnnotation",
        schema = annotation_schema,
        strict = TRUE
      )
    )
  )
)

file <- foundry_file_upload(jsonl, purpose = "batch")
unlink(jsonl)
batch <- foundry_batch_create(file$file_id, endpoint = "/v1/responses")
foundry_batch_get(batch$batch_id)
```

The Batch API is the right choice when the schema is stable and the job is large
enough that lower cost and asynchronous execution matter more than immediate
feedback.

## Embed responses for clustering and near-duplicate checks

Embeddings turn text into numeric vectors. For open-ended survey data, use them
to find near-duplicate answers, cluster themes that were not in the original
codebook, or build semantic search over the responses.

```{r embeddings}
embeddings <- foundry_embed(
  responses$response,
  model = "text-embedding-3-small"
)

similarity <- foundry_similarity(embeddings)
head(similarity, 10)
```

High-similarity pairs are useful audit targets. They can reveal duplicate
responses, repeated complaints, or places where the schema splits similar
answers into different labels.

## Validate generated findings with groundedness

After coding and embedding, a researcher often writes a summary. Treat that
summary as a claim and check it against the source responses.

```{r groundedness}
finding <- paste(
  "Students generally praised clear instruction and practical examples.",
  "Several asked for more examples and more support before assessments."
)

grounding_text <- paste(responses$response, collapse = "\n")

groundedness <- foundry_groundedness(
  text = finding,
  grounding_sources = grounding_text,
  query = "What did students say about the course?",
  task = "QnA"
)

groundedness
```

If `grounded` is `FALSE`, inspect `ungrounded_segments` before sharing the
finding. This does not replace human review, but it gives you an auditable check
inside the same R workflow.

## Review table

```{r review-table}
review <- coded |>
  select(
    respondent_id,
    response,
    sentiment,
    primary_theme,
    needs_followup,
    short_summary
  )

review
```

The rendered table and chart below summarize the same extraction results when
the suggested `gt` and `ggplot2` packages are installed.

```{r review-table-rendered, echo = FALSE, eval = run_api && requireNamespace("gt", quietly = TRUE)}
review |>
  dplyr::mutate(
    needs_followup = ifelse(needs_followup, "Review", "No review"),
    response = substr(response, 1, 70)
  ) |>
  gt::gt() |>
  gt::cols_label(
    respondent_id = "ID",
    response = "Response",
    sentiment = "Sentiment",
    primary_theme = "Theme",
    needs_followup = "Follow-up",
    short_summary = "Summary"
  ) |>
  gt::tab_header(title = "Structured annotations from open-ended responses") |>
  gt::tab_options(table.font.names = "Inter")
```

```{r sentiment-theme-chart, echo = FALSE, eval = run_api && requireNamespace("ggplot2", quietly = TRUE), fig.alt = "Bar chart of survey response themes by sentiment."}
theme_counts <- review |>
  dplyr::count(primary_theme, sentiment, name = "responses")

ggplot2::ggplot(
  theme_counts,
  ggplot2::aes(x = primary_theme, y = responses, fill = sentiment)
) +
  ggplot2::geom_col(position = "dodge", width = 0.72) +
  ggplot2::scale_fill_manual(
    values = c(
      negative = "#D13438",
      mixed = "#FFB900",
      positive = "#107C10"
    )
  ) +
  ggplot2::labs(
    title = "Extracted labels make response patterns countable",
    x = "Primary theme",
    y = "Responses",
    fill = "Sentiment"
  ) +
  ggplot2::theme_minimal(base_size = 12) +
  ggplot2::theme(
    legend.position = "bottom",
    panel.grid.minor = ggplot2::element_blank()
  )
```

The workflow leaves a trail: raw response, extracted labels, model metadata,
embedding similarity, and groundedness checks. That trail is the reason foundryR
returns tibbles instead of hiding results inside client objects.

```{r cleanup, include = FALSE}
if (run_api) {
  httptest2::end_vignette()
}
```