deepreinforce-ai

/

Ornith-1.0-35B-GGUF

like 948

Follow

DeepReinforce 1.91k

havent tried it myself yet, but in terms of template, since theyre using the qwen parsers from what ive seen -> try the FROGGERIC v21.3 QWEN 3.5\3.6 template

Well, I've switched to llama.cpp and once running, I have it this prompt and I am now testing... lets see how slow\fast\confident it turns out to be:

You are a coding agent running via pi.dev connected to a local llama.cpp server. Stop writing conversational fillers, introductions, or essays. Perform a comprehensive system discovery, optimization rewrite, and alignment audit immediately using your bash and file primitives.

Execute the following sequential steps:

1. DISCOVER ENVIRONMENT: Look up your host hardware architecture (OS, CPU/GPU, total memory), active llama.cpp endpoint health (via `curl 127.0.0.1:8080/health` or `/v1/models`), and current log parameters at `~/llm-models/llama-server.log` to read active threads, flash attention, and context configurations. CRITICAL: You must execute real-time terminal commands and read these files directly right now; do NOT extrapolate, guess, or pull this information from your prior training memory, conversation history, or cached context.
2. EVALUATE PI.DEV CONFIG: Inspect your own local pi.dev configuration files (such as AGENTS.md, APPEND_SYSTEM.md, or active config JSONs) to evaluate your current context limits, temperatures, and token parameters.
3. OPTIMIZE LAUNCHAGENT: Based purely on the discovered host hardware limits and the capabilities of the loaded model, calculate the optimal server flags to maximize local coding performance. Directly apply these settings by editing or overwriting the user's file at:
   ~/Library/LaunchAgents/com.me.llama-server.plist
   Ensure the `<array>` block splits every flag, value, and file path into its own clean, sequential `<string>` entry so Launchd parses them flawlessly. Keep existing ports and model weights paths identical unless hardware limitations absolutely force a change.
4. VALIDATE & LINT: Execute `plutil -lint ~/Library/LaunchAgents/com.me.llama-server.plist` via bash to confirm the structural validity of your changes.

Output your final response using ONLY the following structured format:

### System Alignment Matrix
| Parameter | llama.cpp Server Value | pi.dev Configuration | Alignment Status (Match/Mismatch) |
| :--- | :--- | :--- | :--- |
| Model Path / Name | | | |
| Context Window (-c) | | | |
| Flash Attention | | | |
| Thread / Batch Count | | | |

*Note: If any metric remains hidden from your runtime inspection tools, explicitly label it [Unverified].*

### Deployment Execution
* **Daemon Reload Command:** [Provide the exact bash command line sequence required to unload, reload, and restart this specific launch agent plist]
* **Optimization Summary:** [A single-sentence summary detailing exactly which performance parameters were rewritten to align the environments]

me~% launchctl print gui/$(id -u)/com.me.llama-server                                   
gui/501/com.me.llama-server = {
    active count = 1
    path = /Users/me/Library/LaunchAgents/com.me.llama-server.plist
    type = LaunchAgent
    state = running

    program = /usr/bin/script
    arguments = {
        /usr/bin/script
        -q
        /dev/null
        /Users/me/dev/llm/llama.cpp/build/bin/llama-server
        -m
        /Users/me/llm-models/ornith-1.0-35b-Q4_K_M.gguf
        --host
        127.0.0.1
        --port
        8080
        -c
        131072
        -ngl
        -1
        --flash-attn
        on
        -t
        10
        -tb
        10
        -ub
        256
        --parallel
        1
        --cont-batching
        --cache-type-k
        q4_0
        --cache-type-v
        q4_0
        --mlock
        --prio
        2
        --prio-batch
        2
        --log-disable
        --no-perf
        --temp
        0.1
        --repeat-penalty
        1.1
        --jinja
        --reasoning-preserve
        --webui
    }

    stdout path = /Users/me/llm-models/llama-server.log
    stderr path = /Users/me/llm-models/llama-server.error.log
    inherited environment = {
        SSH_AUTH_SOCK => /var/run/com.apple.launchd.fmasdfdadsfad/Listeners
    }

    default environment = {
        PATH => /usr/bin:/bin:/usr/sbin:/sbin
    }

    environment = {
        OSLogRateLimit => 64
        XPC_SERVICE_NAME => com.me.llama-server
    }

    domain = gui/501 [100027]
    asid = 100027
    minimum runtime = 10
    exit timeout = 5
    runs = 1
    pid = 42612
    immediate reason = speculative
    forks = 1
    execs = 1
    initialized = 1
    trampolined = 1
    started suspended = 0
    proxy started suspended = 0
    checked allocations = 0 (queried = 1)
    checked allocations reason = no host
    checked allocations flags = 0x0
    last exit code = (never exited)

    resource coalition = {
        ID = 11160
        type = resource
        state = active
        active count = 1
        name = com.me.llama-server
    }

    jetsam coalition = {
        ID = 11161
        type = jetsam
        state = active
        active count = 1
        name = com.me.llama-server
    }

    spawn type = daemon (3)
    jetsam priority = 40
    jetsam memory limit (active) = (unlimited)
    jetsam memory limit (inactive) = (unlimited)
    jetsamproperties category = daemon
    submitted job. ignore execute allowed
    jetsam thread limit = 32
    cpumon = default

    properties = keepalive | runatload | inferred program
}
me~% 

{
  "providers": {
    "llamacpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "llamacpp",
      "models": [
        {
          "id": "ornith-1.0-35b-Q4_K_M",
          "name": "Ornith 1.0 35B Q4_K_M (Local)",
          "reasoning": true,
          "contextWindow": 131072,
          "maxTokens": 8192,
          "maxHistoryTokens": 24000,
          "thinkingLevelMap": {
            "off": "none",
            "minimal": "minimal",
            "low": "low",
            "medium": "medium",
            "high": "high"
          },
          "compat": {
            "supportsDeveloperRole": false,
            "supportsUsageInStreaming": true
          }
        },
        {
          "id": "ornith-1.0-35b-Q5_K_M",
          "name": "Ornith 1.0 35B Q5_K_M (Local)",
          "reasoning": true,
          "contextWindow": 131072,
          "maxTokens": 8192,
          "maxHistoryTokens": 24000,
          "thinkingLevelMap": {
            "off": "none",
            "minimal": "minimal",
            "low": "low",
            "medium": "medium",
            "high": "high"
          },
          "compat": {
            "supportsDeveloperRole": false,
            "supportsUsageInStreaming": true
          }
        },
        {
          "id": "ornith-1.0-35b-Q6_K",
          "name": "Ornith 1.0 35B Q6_K (Local)",
          "reasoning": true,
          "contextWindow": 131072,
          "maxTokens": 8192,
          "maxHistoryTokens": 24000,
          "thinkingLevelMap": {
            "off": "none",
            "minimal": "minimal",
            "low": "low",
            "medium": "medium",
            "high": "high"
          },
          "compat": {
            "supportsDeveloperRole": false,
            "supportsUsageInStreaming": true
          }
        }
      ]
    }
  }
}