(DRAFT) Restructuring semantic_text field type page #138571
Conversation
|
Pinging @elastic/core-docs (Team:Docs) |
🔍 Preview links for changed docs |
ℹ️ Important: Docs version tagging👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version. We use applies_to tags to mark version-specific features and changes. Expand for a quick overviewWhen to use applies_to tags:✅ At the page level to indicate which products/deployments the content applies to (mandatory) What NOT to do:❌ Don't remove or replace information that applies to an older version 🤔 Need help?
|
There was a problem hiding this comment.
This is looking really nice already, I left some comments and ideas, but particularly the how-to ideas might be best left until SMEs have had the time to digest the initial proposal of breaking these pages up, which will have to wait until next week.
I think the landing page feedback is probably actionable right now though if you want to rework that a bit :)
| serverless: ga | ||
| --- | ||
|
|
||
| # How-to guides for `semantic_text` |
There was a problem hiding this comment.
High-level I think we can organize the how-to's into 3 clear categories:
- Setup and configuration
- Ingestion
- Search and retrieval
Maybe they could be subpages, but please don't feel obliged to jump on this immediately and also of course feel free to push back if that sounds like overkill. Here's the overview of what that might look like, and note that it also implies moving a couple of things out of the reference section which might belong together more naturally under how-to. But this might require additional refactoring that isn't worth the ROI immediately. So just take this as food for thought :)
Expand to see what a potential how-to subpages restructuring would look like
# How-to guides for `semantic_text`
## Setup & configuration
### Configure inference endpoints
- Use default and preconfigured endpoints
- Use ELSER on EIS
- Use a custom inference endpoint
- Use dedicated endpoints for ingestion and search
- **[MOVED FROM REF]** Inference endpoint validation
## Ingestion
### Index pre-chunked content
- Disable automatic chunking
- Index documents
### Use copy_to and multi-fields with semantic_text
- Use copy_to
- Use multi-fields
### **[MOVED FROM REF]** Update documents with semantic_text fields
- Full document updates (examples)
- Partial updates using Bulk API (examples)
- Partial updates using Update API (examples)
- Scripted updates
## Search & retrieval
### **[MOVED FROM REF]** Query semantic_text fields
- Using match queries
- Using kNN queries
- Using sparse vector queries
- Using semantic queries (legacy)
### Retrieve indexed chunks
### Return semantic_text field embeddings
- Return semantic field embeddings in _source
- Return semantic field embeddings using fields
### Highlight the most relevant fragments
- Highlight semantic_text fields
- Enforce semantic highlighter
- Retrieve fragments in original order
### Perform cross-cluster search (CCS) for semantic_text
There was a problem hiding this comment.
I like this organization idea! I'll wait for SME feedback before implementing. If they agree, I'll reorganize the how-to content accordingly.
leemthompo
left a comment
There was a problem hiding this comment.
Overview page is looking good! I have a few language suggestions, and a few optional nits at this stage :)
| 1. In this example, there is no `inference_id` is specified, so `semantic_text` uses a [default inference endpoint](./semantic-text-how-tos.md#default-and-preconfigured-endpoints). | ||
|
|
||
| ::: | ||
| For a complete walkthrough on how to perform semantic search with `semantic_text` fields, refer to the [Semantic search with semantic_text](docs-content://solutions/search/semantic-search/semantic-search-semantic-text.md) tutorial. |
There was a problem hiding this comment.
nit: I would make this a TIP for purely visual appeal
| the [Create {{infer}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put) | ||
| to create the endpoint. If not specified, the {{infer}} endpoint defined by | ||
| `inference_id` will be used at both index and query time. | ||
| The `semantic_text` field type documentation is organized into reference content and how-to guides. |
There was a problem hiding this comment.
I'd use subheadings here to demarcate the Reference and How-to lists in the overview, makes it easier to scan the page via the OTP too :)
| : (Optional, object) Specifies the index options to override default values | ||
| for the field. Currently, `dense_vector` and `sparse_vector` index options are supported. | ||
| For text embeddings, `index_options` may match any allowed. | ||
| The [Reference](./semantic-text-reference.md) page provides parameter descriptions, inference endpoint configurations, chunking behavior, querying options, limitations, and other technical specifications, for instance: |
There was a problem hiding this comment.
This lead-in sentence is potentially overly detailed, considering the list that follows repeats much of the same info. I'd just make it a shorter and more general lead-in.
| documents are ingested into the index. When the first document is indexed, the | ||
| `inference_id` will be used to generate underlying indexing structures for the | ||
| field. | ||
| The [How-to guides](./semantic-text-how-tos.md) page contains procedure descriptions and examples for configuring inference endpoints, pre-chunking content, retrieving embeddings, highlighting fragments, cross-cluster search, and other common tasks, for instance: |
There was a problem hiding this comment.
Same comment about the lead-in sentence
| ::::{warning} | ||
| When updating an `inference_id` it is important to ensure the new {{infer}} endpoint produces embeddings compatible with those already indexed. This typically means using the same underlying model. | ||
| :::: | ||
| 1. In this example, there is no `inference_id` is specified, so `semantic_text` uses a [default inference endpoint](./semantic-text-how-tos.md#default-and-preconfigured-endpoints). |
There was a problem hiding this comment.
| 1. In this example, there is no `inference_id` is specified, so `semantic_text` uses a [default inference endpoint](./semantic-text-how-tos.md#default-and-preconfigured-endpoints). | |
| 1. In this example, the `semantic_text` field uses a [default inference endpoint](./semantic-text-how-tos.md#default-and-preconfigured-endpoints) because the `inference_id` parameter isn't specified. |
| ```{applies_to} | ||
| serverless: ga | ||
| ``` | ||
| The following example creates an index mapping with a `semantic_text` field: |
There was a problem hiding this comment.
| The following example creates an index mapping with a `semantic_text` field: | |
| ## Basic `semantic_text` mapping example | |
| The following example creates an index mapping with a `semantic_text` field: |
totally optional, very in the weeds here 😄, but maybe it would be nice to set the example off under its own subheading
| 1. In this example, there is no `inference_id` is specified, so `semantic_text` uses a [default inference endpoint](./semantic-text-how-tos.md#default-and-preconfigured-endpoints). | ||
|
|
||
| ::: | ||
| For a complete walkthrough on how to perform semantic search with `semantic_text` fields, refer to the [Semantic search with semantic_text](docs-content://solutions/search/semantic-search/semantic-search-semantic-text.md) tutorial. |
There was a problem hiding this comment.
| For a complete walkthrough on how to perform semantic search with `semantic_text` fields, refer to the [Semantic search with semantic_text](docs-content://solutions/search/semantic-search/semantic-search-semantic-text.md) tutorial. | |
| For a complete example, refer to the [Semantic search with `semantic_text`](docs-content://solutions/search/semantic-search/semantic-search-semantic-text.md) tutorial. |
keeps the sentence concise, because the main info is already embedded in the link text
| content using an inference endpoint. Long passages | ||
| are [automatically chunked](#auto-text-chunking) to smaller sections to enable | ||
| the processing of larger corpuses of text. | ||
| The `semantic_text` field type simplifies [semantic search](docs-content://solutions/search/semantic-search.md) by providing defaults for the required infrastructure. Using `semantic_text`, you don't have to manually configure mappings, set up ingestion pipelines, or handle chunking. The field type automatically: |
There was a problem hiding this comment.
| The `semantic_text` field type simplifies [semantic search](docs-content://solutions/search/semantic-search.md) by providing defaults for the required infrastructure. Using `semantic_text`, you don't have to manually configure mappings, set up ingestion pipelines, or handle chunking. The field type automatically: | |
| The `semantic_text` field type simplifies [semantic search](docs-content://solutions/search/semantic-search.md) by providing sensible defaults that automate most of the manual work typically required for vector search. Using `semantic_text`, you don't have to manually configure mappings, set up ingestion pipelines, or handle chunking. The field type automatically: |
There was a problem hiding this comment.
This isn't perfect but I the original doesn't read exactly right to me, feel free to tweak of course.
📷 Preview
Semantic text field type
semantic_textfield type referencesemantic_textSummary
This PR restructures and refines the
semantic_textfield type page.Main changes
1. Content and wording refinements
2. Content restructuring
Restructured the
semantic_textdocumentation into three focused pages:Main page (
semantic-text.md): Converted to an overview page with an introduction explaining whatsemantic_textis and an overview section linking to the reference and how-to pages.Reference page (
semantic-text-reference.md): New dedicated reference page containing parameter descriptions, inference endpoint configurations, chunking behavior, update operations, querying options, and limitations.How-to guides page (
how-to-semantic-text.md): New dedicated how-to page containing procedure descriptions and examples for common tasks, including configuring inference endpoints, pre-chunking content, retrieving embeddings, highlighting fragments, and cross-cluster search.Our main reasons for splitting the docs this way:
Feedback and suggestions are welcome!
Related issue: elastic/docs-content#3836