a
/
badguardhome
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
2,456 Commits 3 Branches 0 Tags 192 MiB
Commit Graph

50 Commits

Author SHA1 Message Date
Simon Zolin 4efc464e98 - querylog: file rotation didn't work properly; fix entry searching algorithm 
If AGH is restarted, file rotation timer is reset
which can lead to the situation when file rotation procedure is never started.

Squashed commit of the following:

commit 427ae91a512cd146ebfffad06ed24eb723cb9e7d
Merge: 067fac65 e56c746b
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Wed Sep 2 18:18:46 2020 +0300

    Merge remote-tracking branch 'origin/master' into qlogs-rotate

commit 067fac65b1a87d499900f4860ffa96ed8208967c
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Wed Sep 2 15:30:48 2020 +0300

    minor

commit c2059a15700e5696cb1bb5cd49129c6020d986f4
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Wed Sep 2 14:53:07 2020 +0300

    improve

commit a279438eaf1cf40b820652093fb56d56784de7d8
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Tue Sep 1 18:49:14 2020 +0300

    minor

commit 26ac130f139f565de39200e484b3bd4a04afcfcc
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Tue Sep 1 13:54:27 2020 +0300

    rename

commit 0fad7b88dbeadcddd4d77536a18da72f3203ea80
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Tue Sep 1 13:05:36 2020 +0300

    + TestQLogSeek

commit fa6afc6d4dc592b1fef67c4a069ea50fae600a58
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Tue Sep 1 13:05:34 2020 +0300

    minor

commit 11e6ab9131e5c37467e8530a2db95a82bbb0603b
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Mon Aug 31 19:45:47 2020 +0300

    fix tests

commit 7cbb89948df0e69b1bae8f8cde1879b5b1c4b1d6
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Mon Aug 31 19:29:43 2020 +0300

    - querylog: fix entry searching algorithm

commit 745d44863d88b321bd7001f24a68620f7ef05819
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Mon Aug 31 18:34:14 2020 +0300

    - querylog: file rotation didn't work properly

    If AGH is restarted, file rotation timer is reset
     which can lead to the situation when file rotation procedure is never started.
 2020-09-02 19:42:26 +03:00
Artem Baskal 6b61429572 Pull request 743: + client: Query Logs Infinite Scroll 
Merge in DNS/adguard-home from feature/infinite_scroll_query_logs to master

Squashed commit of the following:

commit 4407ef2e7c055066257da791fbd65e6b0a495729
Merge: 40b74522 0a4781be
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Tue Sep 1 16:20:23 2020 +0300

    Merge branch 'master' into feature/infinite_scroll_query_logs

commit 40b745225112cf8d664220ed8f484b0aa16e997c
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Tue Sep 1 15:46:27 2020 +0300

    Remove dynamic translation of toasts

commit f08fa7b8c6a243f6b10e924aebccc183ce7814fd
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Tue Sep 1 13:59:53 2020 +0300

    Remove renderLimitIdx, update isEntireLog

commit 0f1b02616faaa5759c0a3f6d8257117fa22094d9
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Tue Sep 1 11:11:14 2020 +0300

    Rename variables

commit 0928570c689c1fa704af775382620d68893e7c1c
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Tue Sep 1 11:06:50 2020 +0300

    Make query logs short polling function more expressive

commit 9e773cbd6c287a1c799fa2680f3462508462ea7a
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Tue Sep 1 11:06:19 2020 +0300

    Fix Toast translation interface

commit f9c57033e5adc5788954cf086b2f114dd8938bcb
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Mon Aug 31 17:01:36 2020 +0300

    Do not hide loader

commit b86ba48613437f5559a748ad9aa4cf79d15db082
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Mon Aug 31 16:56:34 2020 +0300

    Add dynamic translation for all toasts

commit b9d1d9b447ca90a3c179e503fa5d4abd3516321e
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Mon Aug 31 16:39:29 2020 +0300

    Prevent getting query logs recursion if query is not changed

commit e25189749f7912648cca4503cfa8d0ad898c4bb6
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Mon Aug 31 10:13:20 2020 +0300

    Decrease page limit to 20

commit 8b248ac5276899de838abf2dc9a69e47599cfc12
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Fri Aug 28 18:47:12 2020 +0300

    Return checkFilteredLogs

commit bf2d65c4a3dca0da6b15f632ae11042b7c8e2045
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Fri Aug 28 18:33:51 2020 +0300

    Review changes

commit 01b5250f9d9136a1f334086d3e2f00d1a928b37b
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Fri Aug 28 15:29:59 2020 +0300

    Remove checkFilteredLogs

commit 25b364c41e6a1489d930c8b3b39b1ab43723f29d
Merge: 1dc66034 2c666cbd
Author: Andrey Meshkov <am@adguard.com>
Date:   Fri Aug 28 14:28:47 2020 +0300

    Merge branch 'feature/infinite_scroll_query_logs' of ssh://bit.adguard.com:7999/dns/adguard-home into feature/infinite_scroll_query_logs

commit 1dc6603421cde9847e792bfe77ff6546e53fbc2a
Author: Andrey Meshkov <am@adguard.com>
Date:   Fri Aug 28 14:28:01 2020 +0300

    disregard maxFileScanEntries only if offset is set

commit bad741ed7f1dccf6959d43d000b8c0150f526f9e
Author: Andrey Meshkov <am@adguard.com>
Date:   Fri Aug 28 11:57:45 2020 +0300

    Fix search behavior when limit is specified

commit 2c666cbdde465cf17434126830dd99ceedfc4cbc
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Thu Aug 27 18:50:28 2020 +0300

    Hide table ref loader during data loading

commit 8b4f7fe642ef9e87a979813dcdbd7817d64c27f9
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Thu Aug 27 18:43:24 2020 +0300

    Repair search

commit 26fae1ae01a789999b8a2181d60b35663a20460a
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Thu Aug 27 17:59:27 2020 +0300

    Resetting initial render index, change loader position on search

commit e2c97ae1a288438267eef9aec71b979319674a71
Author: ArtemBaskal <a.baskal@adguard.com>
Date:   Thu Aug 27 16:02:03 2020 +0300

    Change isScrolledIntoView

... and 32 more commits
 2020-09-01 16:30:30 +03:00
Simon Zolin 21dfb5ffe8 * querylog: case-insensitive search by domain name 
Close #1828

Squashed commit of the following:

commit 347aff1f8c5293bd06ecc0693970fe447b6e2c75
Merge: f07c8dc0 9df7aba9
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Fri Jul 3 12:10:02 2020 +0300

    Merge remote-tracking branch 'origin/master' into 1828-qlog-case-insens

commit f07c8dc0665585ce00c415293c47c5a8c9724881
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Fri Jul 3 12:09:49 2020 +0300

    minor

commit 512c7acd48f649a0d11b4fef66a1351fcd19635b
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Thu Jul 2 19:01:43 2020 +0300

    minor

commit 9f5700af56340c38a2d892101e32959702816205
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Thu Jul 2 16:53:56 2020 +0300

    * querylog: case-insensitive search by domain name
 2020-07-03 12:21:09 +03:00
Andrey Meshkov b1c951fb2c Merge: + new query logs API 
* commit '59c4a2886a97143e3a36912ec895dc1a06be88cc':
  openapi
  If there are no more older entries, `"oldest":""` is returned.
  fix search by "whitelisted", "rewritten"
  doc
  fix whois test
  + "dot"
  openapi
  * minor
  + client_proto
  * openapi
  + new query logs API
 2020-06-18 00:23:52 +03:00
Simon Zolin bbc4eda39e If there are no more older entries, `"oldest":""` is returned. 2020-06-16 12:10:17 +03:00
Simon Zolin 15e714350c fix search by "whitelisted", "rewritten" 2020-06-16 11:47:26 +03:00
Simon Zolin 1b5fb97a02 + querylog,stats: anonymize client ip: /16 2020-06-03 19:08:11 +03:00
Simon Zolin bccb939346 + "dot" 2020-05-29 11:46:37 +03:00
Simon Zolin 0904eeffa8 + client_proto 2020-05-29 11:15:22 +03:00
Simon Zolin 0500aa9591 + new query logs API 
+ "upstream"
+ filteringStatusProcessed
+ ctDomainOrClient
remove ctQuestionType, ctDomain, ctClient
 2020-05-28 17:14:50 +03:00
Simon Zolin 32d1f385ff + config: new setting "querylog_file_enabled" 
Close #876

Squashed commit of the following:

commit f83f60a7340d8a3f6de7ecfebb426e47d19e83d8
Merge: cfb72869 391e6199
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Thu May 28 15:17:21 2020 +0300

    Merge remote-tracking branch 'origin/master' into 876-logs

commit cfb72869f7cf0bf59a478ab8c7920c273e2fa5f9
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Thu May 28 12:50:02 2020 +0300

    tests

commit 35376e4f450cf66507d733c931b7ed27eff1f36c
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Wed May 27 18:15:12 2020 +0300

    fix

commit 0cfb802d73db52a4b09c459a68a8a18918447b76
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Wed May 27 16:49:52 2020 +0300

    tests

commit 03ca280b6aed3a4880a9d4f4cd18bf47b1c742f6
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Wed May 27 15:32:27 2020 +0300

    + config: new setting "querylog_file_enabled" - query log will be written to a file
 2020-05-28 15:29:36 +03:00
Andrey Meshkov 61a24ffc71 *(querylog): decode methods to a new file 2020-05-26 16:24:31 +03:00
Andrey Meshkov ceedc2d1ff *(querylog): added offset/limit parameters 
Actually, this is a serious refactoring of the query log module. The
rest API part is refactored, it's now more clear how the search is
conducted. Split the logic into more files and added more tests.

Closes: https://github.com/AdguardTeam/AdGuardHome/issues/1559
 2020-05-26 15:37:37 +03:00
Simon Zolin 93a597fce4 - querylog: fix get data 
Close #1589

Squashed commit of the following:

commit 83d1d15f06a67746feb0e82fb71d5ea02f86ab69
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Fri Apr 17 14:11:00 2020 +0300

    - querylog: fix get data
 2020-04-17 14:40:13 +03:00
Andrey Meshkov e96fd6e42d -: fix qlog test on Windows 2020-04-15 14:11:31 +03:00
Simon Zolin 2e845e4f4d + qlog: hide_client_ip setting 2020-03-20 14:50:43 +03:00
Simon Zolin bebd43923b - querylog: querylog_memsize setting was reset to 0 
Since v0.100.0 the query logs memory buffer was not active,
 despite the default capacity of 1000 items.

Now we remove querylog_memsize setting and replace it with querylog_size_memory
 2020-03-19 15:53:25 +03:00
Simon Zolin a5c2cdaf38 + dnsforward: add dnssec_enabled option 2020-03-18 18:05:02 +03:00
Andrey Meshkov f84331abde Merge: -(dnsforward): fixing an issue with the querylog speed 
Squashed commit of the following:

commit 7db062d2ce40369fc6c7eaa89f678c660d531516
Author: Andrey Meshkov <am@adguard.com>
Date:   Thu Feb 27 12:08:58 2020 +0300

    *(dnsforward): added comment about oldest

commit dbec84d8a8c60ca040faa59f8aa818ed5959e92a
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Thu Feb 27 11:16:58 2020 +0300

    fix

commit d6df1ee28b3918e8d1676ff6ab4b516d46fe4202
Author: Andrey Meshkov <am@adguard.com>
Date:   Wed Feb 26 20:42:11 2020 +0300

    -(dnsforward): fixing an issue with the querylog speed
 2020-02-27 14:02:05 +03:00
Andrey Meshkov df427b6822 *(dnsforward): fix reading in-memory entries 2020-02-21 16:50:20 +03:00
Andrey Meshkov 9d8a95f836 *(dnsforward): fix in-memory log entries sort orderr 2020-02-21 13:01:41 +03:00
Andrey Meshkov 87c0410914 -(dnsforward): start using lastProbeLineIdx in the Seek 2020-02-21 12:57:12 +03:00
Andrey Meshkov 696a6df6ba *(dnsforward): replaced old search with the new one 2020-02-21 01:07:30 +03:00
Andrey Meshkov 90f2c18353 *(dnsforward): qlogreader implementation 2020-02-20 21:12:51 +03:00
Andrey Meshkov 712023112d *(dnsforward): finished new qlog_file implementation 2020-02-20 19:38:11 +03:00
Andrey Meshkov 67dc7d7d88 *: better tests for qlog 2020-02-20 14:11:33 +03:00
Andrey Meshkov 9d5f201ddf *(dnsforward): added QLogFile struct 
This struct is supposed to use an easier algorithm for scanning query
log files. The end goal is allow us scanning multiple query log files.
 2020-02-18 13:02:59 +03:00
Simon Zolin 6563886b49 Merge: - install: recover from error on DNS server start 
Close #1293

Squashed commit of the following:

commit 0981754c5c2c67f2567ee4af0d9ab24377c53413
Merge: ef81f2c8 a6d75118
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Thu Jan 16 14:19:20 2020 +0300

    Merge remote-tracking branch 'origin/master' into 1293-install

commit ef81f2c886f3bfcff4e4352d7ecea6642be7d8e1
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Mon Dec 30 18:32:11 2019 +0300

    linter

commit 9e205be53d9de25bd2ad63398644e14b09f95238
Author: Simon Zolin <s.zolin@adguard.com>
Date:   Mon Dec 30 17:22:17 2019 +0300

    - install: recover from error on DNS server start

    Close all modules properly
    Don't register HTTP handlers twice
 2020-01-16 14:25:40 +03:00
Simon Zolin 29998a8959 - querylog: incorrect client IP when blocked by hosts filter 2020-01-09 18:27:44 +03:00
Simon Zolin e7727e9f63 + dnsforward: match CNAME with filtering rules 
+ GET /control/querylog: add "cname_match" field

* querylog: Add() now receives an object with parameters
 2019-12-03 17:01:26 +03:00
Simon Zolin 7e45c2fc24 + new setting "dns.querylog_memsize" 
* set 1000 entries by default (not 5000)
 2019-12-02 15:46:03 +03:00
Simon Zolin a31116635e * querylog: delete code for time counter 2019-11-25 15:20:56 +03:00
Simon Zolin 305df63054 - querylog: remove time counter (fix zero div exception) 2019-11-25 13:40:27 +03:00
Simon Zolin 81e8bbe63c * /control/querylog_config: support optional parameters 2019-11-20 20:12:46 +03:00
Simon Zolin a7742a3665 - querylog: fix linter issue 2019-11-19 17:13:12 +03:00
Simon Zolin 12f4ebc6a5 + querylog: preserve searching compatibility with the previous version 2019-11-19 17:09:54 +03:00
Simon Zolin 68cd7976b7 * querylog: add more tests 2019-11-19 15:09:53 +03:00
Simon Zolin 0cd6781a9a * QueryLog.Add() now receives net.IP, not net.Addr 2019-11-19 15:09:53 +03:00
Simon Zolin 2f5d6593f2 * querylog: major refactor: change on-disk format and API 
speed up decoding
speed up search
compatible with previous format (when not searching)
 2019-11-19 15:08:51 +03:00
Simon Zolin 0d4e95b36d * querylog: don't return entries without Question data 2019-11-12 18:14:33 +03:00
Simon Zolin 8323c0c4b6 Merge: * querylog: skip decoding errors 
Close #753

* commit 'c74ae0d0e7ca03a592663025dc644397c1e31d57':
  * querylog: skip decoding errors
 2019-10-24 16:00:07 +03:00
Simon Zolin c047053646 - querylog: writing to a file could stop randomly 2019-10-24 14:28:24 +03:00
Simon Zolin c74ae0d0e7 * querylog: skip decoding errors 
We read line from file and pass it to a JSON decoder.
JSON decoder is now a local object.
 2019-10-22 19:16:04 +03:00
Simon Zolin e2c26ec554 * API changes 
* filtering_info -> filtering/status
* filtering_config -> filtering/config
 2019-10-14 15:55:58 +03:00
Simon Zolin 90db91b0fd * querylog: refactor: move HTTP handlers to querylog/ 2019-10-09 19:38:58 +03:00
Simon Zolin c9b4a0742f * querylog, stats: don't print unnecessary log messages 2019-10-07 12:03:56 +03:00
Andrey Meshkov b835baf013 *(global): CI scripts 2019-10-02 15:41:14 +03:00
Simon Zolin 81828c87c1 * querylog: POST /control/querylog 2019-09-26 18:27:31 +03:00
Simon Zolin adb422fedf filtering: refactor; change API; add "filters_update_interval" setting 
+ config: "filters_update_interval"
* add /control/filtering_info
* remove /control/filtering/enable
* remove /control/filtering/disable

* add /control/filtering_config
* remove /control/filtering/status

* add /control/filtering/set_url
* remove /control/filtering/enable_url
* remove /control/filtering/disable_url
 2019-09-12 18:38:13 +03:00
Simon Zolin 8104c902ee * querylog: move code to a separate package 
+ config: "querylog_interval" setting
/control/querylog_config, /control/querylog_info
+ POST /control/querylog_clear
 2019-09-12 18:35:13 +03:00