trying to parse jellyfin logs
Last active a month ago
44 replies
10 views
- AE
Log: [2023-02-12 00:20:58.684 +01:00] [INF] [48] Jellyfin.Server.Implementations.Users.UserManager: Authentication request for "test" has been denied (IP: "1.1.1.1").
pattern: '[%{TIMESTAMPISO8601:timestamp}].?Authentication request for ".?" has been denied (IP: "%{IP:sourceip}").$'
Tried multiple pattern variations, but failed. Someone got any tips?
- AE
time="12-02-2023 20:16:23" level=info msg="child-A1AD/jellyfin-logs has debug enabled" id=misty-night time="12-02-2023 20:16:23" level=fatal msg="crowdsec init: Failed to load parsers: failed to load parser config : failed to compile node 'A1AD/jellyfin-logs' in '/home/jeroen/hub/.tests/jellyfin-logs/runtime/parsers/s01-parse/jellyfin-logs.yaml' : Failed to compile grok '\\\\[%{TIMESTAMP_ISO8601:timestamp}\\\\].*Authentication request for \"%{DATA:user}\" has been denied \\\\(IP: \"%{IP:source_ip}\"\\\\)': error parsing regexp: invalid character class range:
)-(\n" time="12-02-2023 20:16:23" level=fatal msg="crowdsec init: Failed to load parsers: failed to load parser config : failed to compile node 'A1AD/jellyfin-logs' in '/home/jeroen/hub/.tests/jellyfin-logs/runtime/parsers/s01-parse/jellyfin-logs.yaml' : Failed to compile grok '\\\\[%{TIMESTAMP_ISO8601:timestamp}\\\\].*Authentication request for \"%{DATA:user}\" has been denied \\\\(IP: \"%{IP:source_ip}\"\\\\)': error parsing regexp: invalid character class range:
)-(\n"
ERRO[12-02-2023 20:16:23] running test 'jellyfin-logs' failed: fail to run '/usr/bin/crowdsec -c /home/jeroen/hub/.tests/jellyfin-logs/runtime/config.yaml -type jellyfin -dsn file://jellyfin-logs.log -dump-data /home/jeroen/hub/.tests/jellyfin-logs/results' for test 'jellyfin-logs': exit status 1
?
Do you want to remove runtime folder for test 'jellyfin-logs'? (default: Yes) Yes
────────────────────────
Test Result
────────────────────────
jellyfin-logs ❌
────────────────────────``` - II
Try one escape character then
- II
Also if you copy and paste make sure to paste into a basic text editor first to strip formatting as they may be hidden characters
- AE
len(results["success"][""]) == 0
- AE
ah man, discord is driving me crazy
- AE
so i am using this now:
pattern: '\[%{TIMESTAMP_ISO8601:timestamp}\].*Authentication request for "%{DATA:user}" has been denied \(IP: "%{IP:source_ip}"\)'
- II
I haven't got my laptop to hand I will test it tomorrow it most likely the timestamp as that always trips it up
- AE
ok thanks again, for the help!
- AE
` patternsyntax: TIMESTAMPWORKAROUND: '[%{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:%{MINUTE}:%{SECOND}.%{INT:milliseconds}[T ]%{ISO8601_TIMEZONE}]'
nodes:- grok:
pattern: '%{TIMESTAMPWORKAROUND:timestamp}.*Authentication request for "%{DATA:user}" has been denied (IP: "%{IP:sourceip}")' `
- grok:
- AE
Tried with a workaround for the timestamp but no cigar.
- LE
I'll see if i have time today to have a look
- AE
When i remove the timezone detection i get to export the user and source_ip
- AE
- AE
but still: len(results["success"][""]) == 0
- LE
for the tests can you show me a successful login?
- AE
give me a sec
- AE
[2023-02-13 08:37:50.708 +01:00] [INF] [115] Jellyfin.Server.Implementations.Users.UserManager: Authentication request for "jeroen" has succeeded.
- AE
Do you know a solution for the log names? The logs are created by day: https://github.com/crowdsecurity/hub/issues/651
- LE
i'll create a pr now almost done
- AE
cool! i am interested to see how you did it
- AE
did you add a BF scenario with it?
- LE
yip
- AE
are you going to create a pr before we test it?
- LE
to do the PR i need to do the tests anyway
- LE
you can get the files from the PR if you want to test as well. The tests pass my side in the hubtest
- LE
i'll have to check this, the timestamp parser is being a pain again and using the current time instead of the logs
- LE
@iiamloz not sure but this has been an issue before were dateparse-ench does weird things can i just remove it from the test?
- AE
@Lepresidente so i am still doing something wrong then, tried your pattern:
`
INFO[13-02-2023 09:53:21] Running test 'jellyfin-logs'
WARN[13-02-2023 09:53:29] Assert file '/home/jeroen/hub/.tests/jellyfin-logs/parser.assert' is empty, generating assertion:len(results) == 2
len(results["s01-parse"]["A1AD/jellyfin-logs"]) == 2
results["s01-parse"]["A1AD/jellyfin-logs"][0].Success == false
results["s01-parse"]["A1AD/jellyfin-logs"][1].Success == false
len(results["success"][""]) == 0Please fill your assert file(s) for test 'jellyfin-logs', exiting
` - LE
i'm not sure, you sure you have updated your test config correctly
- LE
@iiamloz you can ignore the following, remembered we just mangle the date to match what dateparse-ench expects
- II
Or if you on the latest version you can provide a date format string
- AE
sooo i commented out the following at te start of all the tests:
- crowdsecurity/syslog-logs
- crowdsecurity/dateparse-enrich
II didnt know they were needed…. now it is working
- AE
It has been an interesting learning curve so far 🙂
- AE
How did you find out you needed to use the RAILS_TIMESTAMP @Lepresidente ?
- AE
in parsers/s01-parse/LePresidente/jellyfin-logs.md is still a reference to emby by the way
- II
In short if you provide a
evt.StrTimeFormat
it will use that as a custom date format for the evt object so you dont have to force the date to a certian way - LE
ok thanks that is good to know
- II
Must be running 1.4.2 or later
- II
@Lepresidente your PR ready for review?
- LE
hi yea should be
- LE
the fail looks to be not related to what i did
- II
Ye I reran the tests it all passing now
- II
@Lepresidente merged
Last active a month ago
44 replies
10 views