TAKpilot: natural language COP control for CloudTAK tactical operators

Tactical operators using CloudTAK – the browser-based front-end to TAK Server – face a consistent friction point: the gap between knowing what they need to do and completing the action in the interface. Placing a contact marker requires a long-press, a menu selection, a CoT type picker, a coordinate entry dialog, a callsign field, and a confirmation tap. Under calm conditions at a desk, this takes fifteen to twenty seconds. Under time pressure, in gloves, in a vehicle on a dirt track, that sequence fragments. Steps get skipped. Coordinates get mis-entered. Markers end up on the wrong grid. The consequence is not just wasted time – it is a degraded common operating picture that the entire unit is navigating by.

TAKpilot is an AI chat copilot embedded directly in CloudTAK that eliminates menu navigation as the primary interaction mode. Operators type – or dictate – what they need: "place a hostile infantry contact at 37U DP 12345 67890, callsign CONTACT-7". TAKpilot translates the phrase into a structured TAK API call, executes it, and shows the operator a real-time card confirming the action and its result. The marker is on the map in under two seconds. No menus, no picker dialogs, no coordinate format hunting.

This article covers how TAKpilot works in practice: the natural language command model, MIL-STD-2525/APP-6 symbol confirmation, vision and file intelligence, streaming tool cards, the Approve/Deny safety gate, and the product's operational deployment with Ukrainian Defense Forces.

The menu-navigation problem in CloudTAK under time pressure

CloudTAK's menu structure reflects the breadth of what TAK Server can do. That breadth is a capability strength and a cognitive load liability at the same moment. The same operator who can fluently navigate CloudTAK during a planning session at a table becomes error-prone when the operational tempo rises. Research in tactical human factors consistently places UI navigation overhead at 30 to 40 percent of total COP management time for experienced operators – not because the software is poorly designed, but because no tree-based menu system matches the speed of natural speech for routine commands.

The problem compounds across operation types. Subscribing to a new data channel requires navigating to the channel management pane, finding the correct channel in a list that may contain dozens of entries, and clicking subscribe. Changing a mission's priority requires opening the mission, clicking edit, scrolling to the priority field, changing the value, and saving. Each of these is a multi-step UI transaction. Over the course of a four-hour operational period, an active operator might perform sixty to eighty such transactions. The cumulative navigation cost is substantial – and it competes directly with the time the operator should spend reading and interpreting the picture rather than managing it.

How natural language eliminates navigation overhead

TAKpilot's command model is built on LLM function calling. Every CloudTAK operation exposed by the TAKpilot tool library is defined as a JSON schema: a function name, a plain-language description of what the function does, and a typed parameter set with validation constraints. When an operator sends a message, the model receives it alongside the complete tool library and selects the appropriate function – or sequence of functions – and populates the parameters from the natural language input.

Representative natural language commands and what they execute:

• "Place a hostile armored vehicle at grid 37U DP 88800 44400, callsign T-72-ALPHA" – resolves "hostile armored vehicle" to the correct MIL-STD-2525 CoT type string, calls place_marker with the MGRS grid and callsign.
• "List all active missions in sector BRAVO" – calls list_missions with a sector filter, returns a formatted table of mission names, priorities, assigned callsigns, and last-modified timestamps.
• "Subscribe me to channel DELTA and unsubscribe from channel ECHO" – chains subscribe_channel and unsubscribe_channel calls, executing both and reporting results in a single response card.
• "Create a logistics mission for 3rd Platoon, priority URGENT, at grid 37U DP 55555 44444, assign to LOG-1" – calls create_mission with category, priority, location, and assignee in a single structured call.
• "What is the last reported position of EAGLE-1?" – calls query_track for EAGLE-1 and returns grid, heading, speed, and timestamp.

The model handles natural-language ambiguity in unit descriptions – "hostile infantry", "friendly logistics truck", "enemy BTR" – and maps them to the correct CoT type strings from the MIL-STD-2525C/APP-6 symbol taxonomy without requiring the operator to know the type code. If the phrase is genuinely ambiguous (multiple plausible CoT types), TAKpilot asks a single clarifying question rather than guessing.

MIL-STD-2525 / APP-6 symbol confirmation

One of the highest-risk moments in natural language map management is the placement of a contact with an incorrect affiliation or unit type. Placing a hostile unit symbol at a grid where a friendly unit operates – even temporarily – can cause a fire control error. TAKpilot addresses this through a symbol confirmation step that is separate from the Approve/Deny gate used for destructive operations.

When the model resolves a natural language unit description to a CoT type string, TAKpilot renders the corresponding NATO symbol SVG inline in the chat – the actual icon the marker will display on the map – alongside the resolved type name in plain language: "I will place a Hostile Ground Mechanized Infantry [symbol icon] at 37U DP 12345 67890. Confirm?" The operator sees the exact symbol before anything is written to the map. Misread affiliations – "friendly" misheard as "enemy" in a dictated command – are caught at this step rather than discovered when the wrong symbol appears on a live COP.

This confirmation is lightweight: a single button press or the word "yes" in the next message. It adds less than two seconds to the placement flow and eliminates the class of errors where natural language ambiguity produces a correctly executed but semantically wrong map update.

Vision and file intelligence

A significant volume of tactical information reaches operators as images: photographs of hand-drawn sketches, scanned situation reports, PDF overlays distributed over email or messaging apps. Manually re-entering this information into CloudTAK is a time-intensive transcription task with a high error rate for grid references and unit symbols. TAKpilot's vision pipeline automates this transfer.

Operators attach a PNG, JPG, or PDF to the chat and send a command such as "process this SITREP" or "extract all contacts from this sketch". TAKpilot passes the file to a vision-capable model – Claude Sonnet or Opus depending on the session configuration – with a structured extraction prompt. The model identifies every map-relevant entity in the image: MGRS grid references, callsigns, unit type symbols (read from MIL-STD-2525 or APP-6 sketch conventions), bearing lines, phase lines, free-text annotations, and no-fire area boundaries. Output is a structured JSON array.

TAKpilot presents the extracted entities in a confirmation card before any map writes occur: "I found 6 entities in your SITREP. Entity 1: hostile mechanized platoon at 37U DP 12345 67890 (confidence 0.94) [symbol]. Entity 2: friendly observation post at 37U DP 11111 22222 (confidence 0.88) [symbol]…" The operator reviews, corrects any misread grids, deselects any entity they do not want to place, and confirms. TAKpilot then executes the placements in parallel. A six-entity SITREP that would require four to six minutes of manual ATAK data entry is on the map in under thirty seconds from confirmation.

Confidence thresholds are enforced: entities below 0.70 confidence are flagged explicitly and the operator must manually confirm their grids before placement. TAKpilot does not silently place low-confidence entities – the validation is visible to the operator, not hidden in an automated pipeline.

Streaming tool cards: real-time audit trail

Every action TAKpilot takes is visible to the operator in real time through streaming tool cards – collapsible panels that appear in the chat as each function call is initiated and completed. A tool card shows the function name, the input parameters as structured JSON, the execution time in milliseconds, and the HTTP response status from CloudTAK. For multi-step operations, each step generates its own card, appearing in sequence as the chain executes.

This transparency serves two purposes. First, it gives operators immediate confirmation that what they intended is what TAKpilot executed – they can read the parameter JSON and verify the grid, callsign, and CoT type before looking at the map. Second, it provides a complete timestamped audit trail that persists in the chat session history. After-action review can reconstruct exactly what was placed, when, by which operator, using which natural language input, and with what execution latency. The log entry format – "user: sgt_kovalenko via TAKpilot – action: create_mission – input: 'create logistics mission for 3rd Platoon URGENT'" – preserves operator attribution end-to-end, distinguishing AI-assisted actions from direct CloudTAK UI actions.

Approve/Deny gate on destructive operations

TAKpilot categorizes all CloudTAK operations into two classes: additive (place marker, create mission, subscribe channel, create data package) and destructive (delete mission, remove track, clear channel, delete data package). Additive operations execute immediately following symbol confirmation where applicable – the operator can reverse them with a follow-up command, which itself goes through the destructive gate. Destructive operations are intercepted before execution and require explicit operator authorization.

The approval gate renders the full scope of the pending destructive action: for a "delete all missions in sector ALPHA" command, the operator sees a list of every mission that will be deleted, rendered with its NATO symbol, mission name, assigned callsign, priority, and last-modified timestamp. The list is not an abstract count – it is the actual records, displayed with the same visual language the operator uses on the map. Operators recognize their own mission data faster in symbol-annotated format than in a plain-text list, which reduces the cognitive load of the confirmation decision and lowers the false-confirmation rate.

Execution requires either typing "confirm" in the chat or clicking the explicit confirmation button in the gate card. Closing the card or sending a different message cancels the pending operation. The gate has no timeout – if the operator does not confirm, the operation never executes, regardless of how the AI interpreted the original command.

Operational deployment with Ukrainian defense forces

TAKpilot has been operationally deployed with Ukrainian Defense Forces units using CloudTAK for COP management. The deployment context provides a concrete test of the product's core claims under conditions – time pressure, communication stress, multi-platform operation across ATAK Android, WinTAK, and CloudTAK – that cannot be fully replicated in a training environment.

The primary operational benefit reported by deployed units is the reduction in time-to-map for SITREP information received from forward observers. Verbal SITREPs over radio are transcribed directly into TAKpilot chat; imagery SITREPs are processed through the vision pipeline. In both cases, the information reaches the COP faster than manual entry, and the confirmation gate catches transcription errors before they become map errors. Units also report using TAKpilot for channel subscription management – particularly when operational areas of responsibility shift and operators need to reconfigure their channel set quickly.

The deployment also validated the model-agnostic architecture. Ukrainian units operating in areas with reliable connectivity use Claude Sonnet via the Anthropic API. Units in forward-edge positions with no reliable internet connectivity have tested locally hosted Llama 3.3 and Qwen 2.5 models on tactical hardware. The switch between model backends is a configuration change, not a system re-deployment – the TAKpilot service restarts pointing at a different model endpoint and continues operating with the same tool library and confirmation behavior.

Open-source architecture and deployment

TAKpilot is released under AGPL-3.0. The full source – Node.js service, CloudTAK UI extension, tool library definitions, model configuration, and deployment documentation – is available under AGPL-3.0. The AGPL license ensures that any modifications deployed as a service remain open, which aligns with the product's intended role in defense ecosystems where interoperability and auditability are institutional requirements.

The model-agnostic design is implemented through an OpenAI-compatible endpoint abstraction layer. Any model that implements the OpenAI chat completions API with function calling – Anthropic's models via their compatibility endpoint, locally hosted models served by Ollama or vLLM, or models hosted on AWS Bedrock or Google Vertex AI – can be configured as the TAKpilot backend without code changes. This enables air-gapped deployments on classified networks, where data cannot leave the enclave, to run TAKpilot with a locally hosted model while maintaining feature parity with cloud-connected configurations.

For NATO partner units and defense integrators building on top of the TAK ecosystem, TAKpilot's open-source codebase provides a starting point for custom tool libraries – additional CloudTAK API verbs, integration with external sensor feeds, or unit-specific workflow automation. The tool definition format is standard JSON schema; adding a new capability requires defining the schema and implementing the execution handler. Commercial support, custom integration development, and operator training are available from Corvus Intelligence at corvusintell.com/takpilot.

How to deploy TAKpilot alongside CloudTAK

The following steps summarize the deployment process for a standard CloudTAK installation. Full documentation is in the repository README and the docs/ directory.

1. Obtain the distribution package – Obtain the TAKpilot distribution package and unpack it onto the CloudTAK host. Verify Node.js 20 LTS. Run npm install.
2. Configure the model backend – copy .env.example to .env. Set your Anthropic API key and MODEL=claude-sonnet-4-6 for cloud deployment, or set OPENAI_BASE_URL to a local inference endpoint for air-gapped operation.
3. Point at CloudTAK – set CLOUDTAK_BASE_URL to your CloudTAK instance. No service account required – TAKpilot uses the operator's session token for all API calls.
4. Start the service – npm start. Verify CloudTAK connection verified in the startup log.
5. Inject the chat panel – follow docs/cloudtak-integration.md to add the TAKpilot sidebar to the CloudTAK UI. The chat icon appears in the toolbar after injection.
6. Verify with a test command – log in to CloudTAK, open the TAKpilot panel, and send "list all active missions". Confirm a tool card appears and results are returned correctly.
7. Configure the Approve/Deny gate and per-node model – review config/gates.json, confirm destructive operations are gated, and set the appropriate model for each node type in your unit SOP.

Frequently asked questions

Related reading: For the broader architecture of TAK-based tactical applications, see CloudTAK API integration patterns. For how ATAK plugins extend the underlying platform, see ATAK plugin development. For the natural language interface design patterns that inform TAKpilot's command model, see AI copilot for tactical apps: how LLM agents control the COP.