Logo-amall

trying to parse jellyfin logs

Last active a month ago

44 replies

10 views

  • AE

    Log: [2023-02-12 00:20:58.684 +01:00] [INF] [48] Jellyfin.Server.Implementations.Users.UserManager: Authentication request for "test" has been denied (IP: "1.1.1.1").

    pattern: '[%{TIMESTAMPISO8601:timestamp}].?Authentication request for ".?" has been denied (IP: "%{IP:sourceip}").$'

    Tried multiple pattern variations, but failed. Someone got any tips?

  • AE

    time="12-02-2023 20:16:23" level=info msg="child-A1AD/jellyfin-logs has debug enabled" id=misty-night time="12-02-2023 20:16:23" level=fatal msg="crowdsec init: Failed to load parsers: failed to load parser config : failed to compile node 'A1AD/jellyfin-logs' in '/home/jeroen/hub/.tests/jellyfin-logs/runtime/parsers/s01-parse/jellyfin-logs.yaml' : Failed to compile grok '\\\\[%{TIMESTAMP_ISO8601:timestamp}\\\\].*Authentication request for \"%{DATA:user}\" has been denied \\\\(IP: \"%{IP:source_ip}\"\\\\)': error parsing regexp: invalid character class range:)-(\n" time="12-02-2023 20:16:23" level=fatal msg="crowdsec init: Failed to load parsers: failed to load parser config : failed to compile node 'A1AD/jellyfin-logs' in '/home/jeroen/hub/.tests/jellyfin-logs/runtime/parsers/s01-parse/jellyfin-logs.yaml' : Failed to compile grok '\\\\[%{TIMESTAMP_ISO8601:timestamp}\\\\].*Authentication request for \"%{DATA:user}\" has been denied \\\\(IP: \"%{IP:source_ip}\"\\\\)': error parsing regexp: invalid character class range:)-(\n"
    ERRO[12-02-2023 20:16:23] running test 'jellyfin-logs' failed: fail to run '/usr/bin/crowdsec -c /home/jeroen/hub/.tests/jellyfin-logs/runtime/config.yaml -type jellyfin -dsn file://jellyfin-logs.log -dump-data /home/jeroen/hub/.tests/jellyfin-logs/results' for test 'jellyfin-logs': exit status 1
    ?
    Do you want to remove runtime folder for test 'jellyfin-logs'? (default: Yes) Yes
    ────────────────────────
    Test Result
    ────────────────────────
    jellyfin-logs ❌
    ────────────────────────```

  • II

    Try one escape character then

  • II

    Also if you copy and paste make sure to paste into a basic text editor first to strip formatting as they may be hidden characters

  • AE

    len(results["success"][""]) == 0

  • AE

    ah man, discord is driving me crazy

  • AE

    so i am using this now:
    pattern: '\[%{TIMESTAMP_ISO8601:timestamp}\].*Authentication request for "%{DATA:user}" has been denied \(IP: "%{IP:source_ip}"\)'

  • II

    I haven't got my laptop to hand I will test it tomorrow it most likely the timestamp as that always trips it up

  • AE

    ok thanks again, for the help!

  • AE

    ` patternsyntax: TIMESTAMPWORKAROUND: '[%{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:%{MINUTE}:%{SECOND}.%{INT:milliseconds}[T ]%{ISO8601_TIMEZONE}]'
    nodes:

    • grok:
      pattern: '%{TIMESTAMPWORKAROUND:timestamp}.*Authentication request for "%{DATA:user}" has been denied (IP: "%{IP:sourceip}")' `
  • AE

    Tried with a workaround for the timestamp but no cigar.

  • LE

    I'll see if i have time today to have a look

  • AE

    When i remove the timezone detection i get to export the user and source_ip

  • AE
  • AE

    but still: len(results["success"][""]) == 0

  • LE

    for the tests can you show me a successful login?

  • AE

    give me a sec

  • AE

    [2023-02-13 08:37:50.708 +01:00] [INF] [115] Jellyfin.Server.Implementations.Users.UserManager: Authentication request for "jeroen" has succeeded.

  • AE

    Do you know a solution for the log names? The logs are created by day: https://github.com/crowdsecurity/hub/issues/651

  • LE

    i'll create a pr now almost done

  • AE

    cool! i am interested to see how you did it

  • AE

    did you add a BF scenario with it?

  • LE

    yip

  • AE

    are you going to create a pr before we test it?

  • LE

    to do the PR i need to do the tests anyway

  • LE

    you can get the files from the PR if you want to test as well. The tests pass my side in the hubtest

  • LE

    i'll have to check this, the timestamp parser is being a pain again and using the current time instead of the logs

  • LE

    @iiamloz not sure but this has been an issue before were dateparse-ench does weird things can i just remove it from the test?

  • AE

    @Lepresidente so i am still doing something wrong then, tried your pattern:

    `
    INFO[13-02-2023 09:53:21] Running test 'jellyfin-logs'
    WARN[13-02-2023 09:53:29] Assert file '/home/jeroen/hub/.tests/jellyfin-logs/parser.assert' is empty, generating assertion:

    len(results) == 2
    len(results["s01-parse"]["A1AD/jellyfin-logs"]) == 2
    results["s01-parse"]["A1AD/jellyfin-logs"][0].Success == false
    results["s01-parse"]["A1AD/jellyfin-logs"][1].Success == false
    len(results["success"][""]) == 0

    Please fill your assert file(s) for test 'jellyfin-logs', exiting
    `

  • LE

    i'm not sure, you sure you have updated your test config correctly

  • LE

    @iiamloz you can ignore the following, remembered we just mangle the date to match what dateparse-ench expects

  • II

    Or if you on the latest version you can provide a date format string

  • AE

    sooo i commented out the following at te start of all the tests:

    • crowdsecurity/syslog-logs
    • crowdsecurity/dateparse-enrich
      II didnt know they were needed…. now it is working
  • AE

    It has been an interesting learning curve so far 🙂

  • AE

    How did you find out you needed to use the RAILS_TIMESTAMP @Lepresidente ?

  • AE

    in parsers/s01-parse/LePresidente/jellyfin-logs.md is still a reference to emby by the way

  • II

    In short if you provide a evt.StrTimeFormat it will use that as a custom date format for the evt object so you dont have to force the date to a certian way

  • LE

    ok thanks that is good to know

  • II

    Must be running 1.4.2 or later

  • II

    @Lepresidente your PR ready for review?

  • LE

    hi yea should be

  • LE

    the fail looks to be not related to what i did

  • II

    Ye I reran the tests it all passing now

  • II

    @Lepresidente merged

Last active a month ago

44 replies

10 views