> ## Documentation Index
> Fetch the complete documentation index at: https://developer.kyberis.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Splunk troubleshooting

> Common errors, per-event status fields, cache and KV Store issues, and where the app logs.

## Configuration and credential errors

These fail the search, or the alert action, with an actionable message rather
than annotating events.

### "No API key is stored for Kyberis credential profile …"

Two distinct causes, called out in the message itself:

1. The profile genuinely has no stored key — add it on the setup page
   (**Apps → Kyberis Threat Intelligence → Set up**).
2. A key exists, but the role running the search lacks the
   `list_storage_passwords` capability — grant it. See
   [Permissions](/integrations/splunk/permissions).

A third cause on hand-edited systems: `default_profile` in `kyberis.conf` points
at a profile that was deleted. The setup page reassigns the default automatically
when a profile is removed, but manual conf edits can still leave it dangling.
Point `default_profile` at an existing profile.

### "base\_url … uses http\://, which would send the API key … unencrypted"

`base_url` must be `https://`, except for local development targets. See
[Network requirements](/integrations/splunk/network#transport-policy). Fix the
URL in `kyberis.conf`.

### "The Kyberis API rejected the configured credentials (HTTP 401/403)"

The key stored for the profile is wrong, revoked, or truncated. Re-save the
profile with a valid key and re-run `| kyberischeck`.

## Per-event status fields

When the search succeeds but individual indicators carry a non-`ok`
`kyberis_status`, see the
[status reference](/integrations/splunk/search-command#kyberis_status-values). In
short:

* `transport_error` — the API was unreachable, and the search also shows a
  warning banner. Check egress, firewall, and DNS from the search head, then
  re-run: these results are never cached, so the next search retries them.
* `plan_limit` — the Kyberis plan ran out mid-search. Partial results are kept
  and the remainder retried next search.
* `error` — the API answered but could not assess that indicator, and
  `kyberis_message` says why.

## Only the first field of `field=a,b` gets enriched

An unquoted comma ends an SPL option value, so `field=src_ip,dest_ip` passes only
`src_ip` to the command. Quote the list: `field="src_ip,dest_ip"`.

## Cache issues

### Warm searches still call the API

Check in order: is `cache=false` set on the search? Is `cache_ttl_seconds = 0`,
which disables the cache? Was the earlier result not `ok` — only `ok` verdicts
are cached? Did `base_url` change, since entries are scoped to it? Finally, check
the per-search cache stats line in `search.log` (`kyberis cache: N hit(s), …`).

### "KV Store … degraded" warnings, or zero cache hits and writes

The cache fails soft, so searches keep working uncached. If the KV Store itself
is down — check **Settings → KV Store status**, or run
`| rest /services/kvstore/status` — note that splunkd does **not** auto-restart a
dead KV Store (mongod) process. A Splunk restart is required to bring it back.

### Non-admin users never write the cache

Expected. Collection write access is admin-only as a security property; see
[Permissions](/integrations/splunk/permissions#the-cache-collection-acl--do-not-broaden-it).
Their searches still read the cache and return full results.

## Search cancellation semantics

Finalizing or cancelling a `| kyberis` search does not abort instantly: the chunk
currently in flight — at most 50 indicators, one API call — drains before the
command exits. Fetched results are kept and cached, and no further calls are
made.

## Logging

* **Search commands** log to the search's own `search.log` (**Job Inspector →
  search.log**). The app ships `default/logging.conf` so its operational lines
  are visible; you can customize the format via `local/logging.conf`.
* **Verbosity** is controlled by `log_level` in `kyberis.conf` — `DEBUG`, `INFO`,
  `WARNING`, or `ERROR`, default `INFO` — for both the commands and the alert
  action. `local/logging.conf` only affects search-command formatting and
  handlers; the alert action does not read it.
* **Credential material** — API keys, `Authorization` headers, session keys — is
  redacted from all log output at every level, including tracebacks.

### Read alert action logs

Alert action output goes to `splunkd.log`, component `sendmodalert`:

```
index=_internal sourcetype=splunkd component=sendmodalert action="kyberis_enrichment"
```

`| sendalert kyberis_enrichment` invocations from the search bar log into that
search's `search.log` instead.

## The alert action does not fire

* Dispatching a saved search via REST with `trigger_actions=1` has been observed
  not to fire alert actions on Splunk 9.4. Test with
  `| sendalert kyberis_enrichment`, or let the scheduler trigger the alert.
* For exit codes and failure policy, see the
  [alert action reference](/integrations/splunk/alert-action#failure-policy).

## The setup page shows raw REST errors on Save

The user is not an admin. Profile setup writes `storage/passwords` and
`kyberis.conf`, which needs admin capabilities — there is no friendlier gate for
non-admins. See [Credential setup](/integrations/splunk/credentials).