Delimiters

Special characters or patterns that separate and structure information in AI prompts and data

Overview

Delimiters are special characters, symbols, or patterns used to separate and organize different parts of text or data in AI systems. They play a crucial role in helping AI models understand the structure of inputs, maintain context, and generate appropriate outputs. From simple comma separators to complex XML-style tags, delimiters enable clear communication between humans and AI.

Types of Delimiters

Basic Delimiters

Single characters (e.g., commas, spaces, semicolons) that separate items in a sequence:

Comma separated: 1,2,3,4,5
Semicolon separated: 1;2;3;4;5
Space separated: 1 2 3 4 5
Pipe separated: first|second|third
Hyphen separated: word-separator
Text Block Delimiters

Quotation marks or triple quotes enclose blocks of text while preserving formatting:

""" (triple quotes)
This is a block of text
that spans multiple lines
and keeps its formatting
"""

''' (single quotes)
Another block of text
with preserved formatting
'''
Structured Delimiters

XML-style tags and other structured formats provide sophisticated organization:

<section>
  Content within XML-style tags
</section>

[block]
Content within bracket tags
[/block]

Common Applications

Input Organization

AI prompts often use custom tags or brackets to clarify who is speaking or what the system should do:

<system>
Be concise and professional
</system>

<user>
Summarize this article
</user>

<content>
Article text goes here...
</content>
Data Structuring

Delimiters organize complex data hierarchies (such as XML or custom tags):

<record>
  <id>12345</id>
  <name>John Doe</name>
  <details>
    <age>30</age>
    <location>New York</location>
  </details>
</record>
Output Formatting

Even model responses can be wrapped to keep sections distinct:

<summary>
Key points from the text
</summary>

<analysis>
Detailed analysis here
</analysis>

<recommendations>
Suggested actions here
</recommendations>

Common Patterns

Hierarchical Structure

Tags or nested blocks specify a hierarchy in the content:

<main>
  <section>
    <subsection>
      Content here
    </subsection>
  </section>
</main>
Sequential Organization

Numbered or labeled segments separate items or sections:

#1: First item
#2: Second item
#3: Third item
---
Next section
---
Final section
Data Separation

Brackets and markers delineate multiple content blocks:

[START_DATA]
Content block 1
[END_DATA]

[START_DATA]
Content block 2
[END_DATA]
Healthcare Applications

Healthcare data can be structured with delimiters for clarity and compliance:

<demographics_and_history>
Patient's name, age, and medical history
</demographics_and_history>

<review_of_symptoms>
    <pertinent_positives>
        <cardiovascular>
            Chest pain, palpitations
        </cardiovascular>
        <respiratory>
            Shortness of breath, cough
        </respiratory>
    </pertinent_positives>
    
    <pertinent_negatives>
        <cardiovascular>
            No syncope, no edema
        </cardiovascular>
        <respiratory>
            No hemoptysis, no wheezing
        </respiratory>
    </pertinent_negatives>
</review_of_symptoms>

<current_medications>
Active medications
</current_medications>

<current_allergies>
Allergies and immunizations
</current_allergies>

Best Practices

Selection Guidelines
  • Choose unambiguous characters
  • Maintain consistency across data sources
  • Consider readability and potential parsing conflicts
  • Enable proper nesting
Implementation Tips
  • Use descriptive tag names
  • Keep structure simple
  • Validate delimiter formatting before processing
  • Handle errors or missing tags gracefully
  • Document conventions
Technical Considerations
  • Escape handling for special characters
  • Nested delimiter management
  • Whitespace handling rules
  • Error recovery strategies
  • Character encoding support
Performance Factors
  • Processing overhead optimization
  • Memory usage efficiency
  • Parsing speed improvements
  • Storage space considerations
  • Search and indexing capabilities