I’ve spent a little more time building with the ML Kit GenAI Prompt API on Android, which has continued to evolve even since I started playing around with it earlier this summer.

Two notable pieces I wanted to touch on were system instructions and structured output. Each is a tool for engineering better prompts and building more specific user experiences.

System Instructions

System instructions help you shape the output & tone of your prompt responses. You can define a style, format, or constraints that are consistent across your dynamic queries.

In my D&D Assistant app I added system instructions to help guide my responses in both tone and format.

val SYSTEM_INSTRUCTION = """  
    You are a Dungeon Master's assistant inside a D&D 5e SRD reference app.
    Every message you receive is one SRD entry. 
    Reply with exactly ONE sentence of at most 30 words, and nothing else.
    Lead with the single most important fact a DM would want at a glance, favoring hard numbers and mechanics:
    damage, AC, HP, saves, DCs, ranges, durations, uses. 
    Never open with "This" or a restatement of the title. 
    No preamble, no markdown, no second sentence — output the sentence alone.
""".trimIndent()

I then used this instruction when generating content with a dynamic prompt.

val prompt = Prompt.userContent(entry) // my dynamically generated prompt
val request = generateContentRequest(  
    systemInstruction = SystemInstruction(Prompt.SYSTEM_INSTRUCTION),  
    text = TextPart(prompt),  
) {  
    temperature = 0.2f  
    candidateCount = 1  
}

This helped guide my prompts in a repeatable way so I could get consistent output across many different rules entries I needed to summarize.

Remember to keep system instructions brief & direct. They should be under 100 tokens and clearly state your persona, output shape, or constraints.

If your use case is complex enough that you’re reaching for the Prompt API over one of the other feature-specific APIs like Summarization, chances are you’ll want to define your output in a repeatable way. System instructions should be a core part of that strategy.

Prefix Caching

If you’ve done some digging into this API you may also have seen the concept of prefix caching, which allows you to define a re-usable prefix for your prompts that get cached for future queries. This is primarily a performance feature, designed to handle large prompt instruction prefixes.

The guidance is to not use system instructions with prefix caching, and instead decide on the right approach for your prompt workflow. Use prefix caching over system instructions when you have to repeat a large (500+ token) prefix on each prompt to optimize for performance.

Structured Output

Text output is great, but the Prompt API is capable of generating all sorts of text, including structured text - like JSON. This is a super powerful aspect of LLMs that enable them to do all sorts of novel things, and is available on device as well via this API.

The structured output API allows you to define an output class for a given prompt. You can define your object structure, including hints for the model, and then provide that along with your prompt.

Let’s define a simple object structure that could be used to convert a monster entry into a summary data model for my summary card.

import com.google.mlkit.genai.schema.annotations.Generable
import com.google.mlkit.genai.schema.annotations.Guide

@Generable("Top level information about a D&D monster")
data class Monster(
    @Guide(description = "The common name of the monster")
    val name: String,
    
    @Guide(description = "The monster type, like undead, fiend, or humanoid.")
    val type: String,
    
    @Guide(
     description = "The monster total HP or hit points.",
     minimum = 0.0,
    )
    val hitPoints: Int,
    
    @Guide(description = "The monster AC or armor class rating.")
    val armorClass: Int,
    
    @Guide(description = "The monster's initiative modifier.")
    val initiative: Int,
    
    @Guide(description = "An optional list of immunities like Acid, Prone, etc.")
    val immunities: List<String>?,
)

Then you can define this output model when prompting. The supported types/constraints cover object types you’d typically deal with.

val promptText = "<Monster block text>"
val baseRequest = GenerateContentRequest.Builder(
	text = TextPart(promptText)
).build()
val typedRequest = generateTypedContentRequest(
    generateContentRequest = baseRequest,
    outputClass = Monster::class, // Define the output model
)

The result is a populated Monster instance, fully populated by the model!

You can do a lot with structured JSON objects. The possibilities are endless for inputting information with well-built prompts, allowing an LLM to process / expand upon them, and generate output models that your app can understand.

Maybe the outputClass is a data model tuned as state for your UI, so that users can input queries to the LLM (via text or app interaction) and the app responds by updating its UI? Ask a question, render a button. I think this could be really powerful!

Overall I’m so excited about the possibility of on-device AI. It makes this technology much more affordable and accessible for smaller apps like the ones I enjoy building.

The Prompt API feels like the true star of the ML Kit GenAI APIs. The features being added continue to support more specific workflows, while giving devs more flexibility on model output. I encourage you to dive deeper and let me know what you’re building on Mastodon or Bluesky!