_ prabs.xyz

On-Device AI : Exposing Apple's Foundation Model on macOS with Vapor (Swift Server)

•4 min read

Apple is losing the AI race to everyone and isn’t innovating enough on the AI front. While other tech giants are moving fast with new AI products and features, Apple still feels like it’s playing catch-up. They really need to step it up if they don’t want to fall too far behind.

At WWDC 2025, Apple announced its Foundation Models framework enabling developers to use Apple's on-device large language models directly in their apps. With this anyone can enable their app to build efficient AI features with privacy and offline support.

Apple's Foundation Model

  • The Foundation Models give developers access to Apple’s on-device model that powers Apple Intelligence, supporting a diverse range of tasks such as summarization, entity extraction, text understanding, dialogue creation, and creative text generation.

  • The framework includes Swift-native APIs and guided generation, allowing developers to generate entire Swift data structures and use tool calling to extend model capabilities for app-specific tasks.

Modeling overview

Why it matters:

  • Runs entirely on device - private, fast.

  • No third-party API keys needed.

  • Can be embedded into your apps.

    Let’s build a simple Swift server using Vapor that exposes the Foundation Model via a REST API. So that we can query it from anywhere on your Mac.

Prerequisites

  • macOS 26 (Tahoe) with Apple Silicon Support

  • Xcode 17+

  • Swift 6+

  • Vapor 4

  • Some Swift familiarity

Step 1: Setup a Vapor Project

brew install vapor
vapor new fm-apple -n
cd fn-apple

In Package.swift, add:

.package(url: "https://github.com/vapor/vapor.git", from: "4.0.0"),

Step 2: Configuring a Vapor Application

Let’s set-up CORS (Cross-Origin Resource Sharing) middleware to allow requests from any origin and supports common HTTP methods and headers.

Finally, register our application's routes.

public func configure(_ app: Application) async throws {

    let corsConfiguration = CORSMiddleware.Configuration(
        allowedOrigin: .all,
        allowedMethods: [.GET, .POST, .PUT, .OPTIONS, .DELETE, .PATCH],
        allowedHeaders: [
            .accept, .authorization, .contentType, .origin, .xRequestedWith, .userAgent,
            .accessControlAllowOrigin,
        ]
    )

    let cors: CORSMiddleware = CORSMiddleware(configuration: corsConfiguration)
    app.middleware.use(cors)

    try routes(app)
}

Step 2: Write a Route for Text Generation

In routes.swift Import foundation models

import Vapor
import FoundationModels

 app.post("generate") { req async throws -> GenerateResponse in

        let body = try req.content.decode(GenerateRequest.self)
        let prompt = body.prompt

        if #available(macOS 26.0, *) {
            let session = LanguageModelSession()
            let response = try await session.respond(to: prompt)
            print("Response: \(response)")
            return GenerateResponse(output: response.content)

        }
        return GenerateResponse(output: "Model not available on this OS version.")

    }

In this snippet, We are creating a create a LanguageModelSession object to call the model. It also provides an way to customize the response generation with the options like instruction, maximumResponseToken, temperature, sampling.

For more info you can check this official documentation

Step 3: Run the Server

Now very minimal version of accessing Foundation Model with specified route is ready. You can just run the server and hit the API end point with you prompt to see the result in action.

Note: Currently system model supports up to 4,096 tokens.

swift run

Now test in terminal:

curl -X POST http://127.0.0.1:8080/generate -H "Content-Type: application/json" -d '{"prompt":"Tell me about Apple foundation models"}

"Apple Foundation Models (AFMs) are a set of large-scale, pre-trained models developed by Apple that are designed to understand and interact with natural language effectively...."

On Privacy Side

  • Everything runs on-device.

  • No data leaves your Mac.

  • Contrast with cloud LLM APIs.

Why Swift for Apple Foundation Models

As languages like Python, Go, Rust continue to dominate in the AI/ML space, Apple is taking a different path by exploring this world with its own language, Swift. It reflects Apple’s preference for building tightly integrated ecosystems, though it remains to be seen how Swift will position itself alongside the more established AI/ML languages.

Apple-First Frameworks

  1. The new Foundation Models framework (introduced in iOS 26 / macOS 26) is only available in Swift / Objective-C APIs right now.

  2. There’s no direct Python or Node binding from Apple. So if you want first-class access, then Swift is the native choice.

Tight Integration with Apple Silicon

  1. Swift can directly talk to Core ML, Neural Engine, and Foundation Models APIs without extra wrappers.

  2. This ensures on-device inference is fast, battery-friendly, and private.

Next steps: I want to try this integrating with any app and building RAG, Tool Calling feature around it. Let’s see how it goes.

© 2025