================================================== Changes from 2019-09-25 00:00:00 +0000 to present. ================================================== ---------------------------------------- version 8.2 at 2021-09-24 21:30:53 +0000 ---------------------------------------- Change: 569436f7847833b3b3b20d15d14f753ab07c61aa Author: Brad Lhotsky Date : 2021-09-24 14:30:36 +0000 Prepping the 8.2 Release Change: ff914b2e69d3bf4c1862ad4970e119ab9a150e12 Author: Brad Lhotsky Date : 2021-09-16 15:57:37 +0000 ES 7.x fixes + bonus * Fix `::Query->add_bool()` for stricter parser in ES 7.x * Add `value_count` as a valid aggregation for sorting Change: 963498bd7d45ba7480a8ef2ba82306bd98f9175c Author: Brad Lhotsky Date : 2021-08-31 20:25:48 +0000 Add --with-missing to es-search.pl If specified, all terms aggregations will calculate the missing element as 'MISSING'. ---------------------------------------- version 8.1 at 2021-09-01 00:43:28 +0000 ---------------------------------------- Change: dc6735a8f9fcb22c3bad773b759c47d4519dc2c1 Author: Brad Lhotsky Date : 2021-08-31 17:43:28 +0000 Setting up the 8.1 release Change: cc2d07e3e2515d02d983dd8dd35fe3a061047a66 Author: Brad Lhotsky Date : 2021-08-31 17:41:12 +0000 Add test for query and minimum_should_match for consistency Change: 5257a66d11394ea8ec140c9e5de0c3e7a8a1d50c Author: Brad Lhotsky Date : 2021-08-31 17:24:45 +0000 Fix tabs which errantly made it into the file Change: 94b47afe216d3361d821ef6585888fed4a6c958b Author: Brad Lhotsky Date : 2021-08-31 17:20:53 +0000 Fix minimum should match handling in the ::Query Change: 365a0a129d40b505bcb8ef3d72809f6e4c75ccfe Author: Brad Lhotsky Date : 2021-08-31 17:18:31 +0000 Update Aggregations documentation * Adds documentation to `::Aggregations` * Adds more docs to `es-aggregate.pl` Change: ce7787fe56b295dc83476f58f9082ade7f0e3661 Author: Brad Lhotsky Date : 2021-08-16 21:02:18 +0000 Starting point for aggregation documentation ---------------------------------------- version 8.0 at 2021-08-14 22:15:28 +0000 ---------------------------------------- Change: 40dbb86b7b8d9b98f4572a9f362fdd94f4d57b40 Author: Brad Lhotsky Date : 2021-08-14 15:15:28 +0000 Support OpenSearch 1.0.0 Add a check to the version lookup for opensearch. If opensearch is discovered, set version to 7.10. ------------------------------------------ version 7.9.2 at 2021-08-14 02:17:04 +0000 ------------------------------------------ Change: 8adedd9d89bdcaac26844547d36813bfc75f511f Author: Brad Lhotsky Date : 2021-08-13 19:17:04 +0000 Fix a warning in the 7.9.1 Release I was unable to `Dist::Zilla` to let me tag a `7.10` release, so I went with `7.9.1`. This caused a warning in `::Connection` for formatting the version string. Change: 84dd4b08f113cadc0d8f71ac7bd5448536910f3f Author: Brad Lhotsky Date : 2021-08-13 19:07:49 +0000 Version number fixed in Markdown ------------------------------------------ version 7.9.1 at 2021-08-14 01:48:01 +0000 ------------------------------------------ Change: 3eef2f64709f5b9643c91544b839a1727b7f32c0 Author: Brad Lhotsky Date : 2021-08-13 18:48:01 +0000 Fix POD for xtests Change: bc51180e9f87730ee0d220e5c1c8b22e7e740253 Author: Brad Lhotsky Date : 2021-08-13 18:38:33 +0000 Aggregations Updated * Recursive aggregation parsing to flatten records into a row/column format * Documentation updates for the `App::ElasticSearch::Utilities::Aggregations` * Make `es-aggregate.pl` more useful generally and move to `scripts/` * Fix alias handling for simple terms aggs Change: d8c8be912a60156290cb68c046e3b3c8fafbc2ae Author: Brad Lhotsky Date : 2021-08-04 17:26:42 +0000 Tests started to the aggregations parser Change: 25e3fc64e388b3a84002d594e68516602970eb89 Author: Brad Lhotsky Date : 2021-08-04 17:13:23 +0000 Suss out the es_expand_aggregate syntax Now looks like: ``` [ALIAS=]AGG:FIELD[:PARAMS] PARAMS := [k=v,k1=v1,v1a] ``` Add a few additional aggregations to the supported list. ---------------------------------------- version 7.9 at 2021-07-30 18:19:09 +0000 ---------------------------------------- Change: f386e7003a442a0419fc62aa30d576312afa9efa Author: Brad Lhotsky Date : 2021-07-30 11:19:09 +0000 Release 7.9 * Fix timeout handling * Allow for connection preferences into the `meta` settings section * Allow `_id` to be shown by `es-search.pl` * Look in `~/.config` for `es-utils.yaml` * Started some work on library for handling aggregation queries Change: ba56db091ef15fd365100e034b17a8de530333e6 Author: Brad Lhotsky Date : 2021-07-14 15:07:21 +0000 WIP: Substantial progress on the multi level aggregation script Change: 5c2517a1264b4a381b11387125e63dada6fcef2a Author: Brad Lhotsky Date : 2021-07-08 14:50:45 +0000 Adding es-aggregate.pl script `es-search.pl` has a lot of cruft. Creating `es-aggregate.pl` as a proof-of-concept starting from a clean slate to see if I can make the code more readable and more reusable. The idea being to push this code back into the library to make working with aggregations easier and cleaner. Change: cfd98ad2e84ee1ea8791debe7f6e1809bd6a9af6 Author: Brad Lhotsky Date : 2021-07-08 07:34:21 +0000 Move the aggregation parsing out to it's own sub This is a long time coming, the handling of aggregations needs a major cleanup. First step is to break things up into more parseable blocks. Change: ea453e9190173af63adce15b684dcad13e3b13c5 Author: Brad Lhotsky Date : 2021-07-08 07:15:14 +0000 Pass timeout to the object creation When converting to `App::ElasticSearch::Utilities::Connection`, I mistakenly removed the pass through of the `--timeout` parameter to the object creation. This restores the timeout capability. Change: c8f664cfb09b928831945f82607eab5830faa066 Author: Brad Lhotsky Date : 2021-07-08 07:14:50 +0000 Allow HTTP username/password in meta The previous commit allowed overriding the destination for the connection. This allows users to override the HTTP basic auth username/password. ```yaml host: es-main.int.example.com meta: access: timestamp: t security: timestamp: @timestamp host: es-security.int.example.com proto: https http-username: bob password-exec: ~/bin/get-es-security-password.sh ``` Change: 7c56813974cc069a6214daef8667551a8d31de65 Author: Brad Lhotsky Date : 2021-07-08 06:55:14 +0000 Added connection details to the meta section This update allows the metadata in the `~/.es-utils.yaml` file to include the `host`, `port`, and `proto`. Example: ```yaml host: es-main.int.example.com port: 80 meta: security: host: es-security.int.example.com port: 443 proto: https ``` Change: 797199f7af436597ffa67c486d5ea275b825e471 Author: Matthew Feinberg Date : 2021-06-24 14:03:07 +0000 For es-search.pl allow _id to be shown as a column when using --show and display _id when showing full result records. Change: 6547b9893ad05e7cef565f4a2caf51b321091de0 Author: Andrei Grechkin Date : 2021-06-11 10:11:34 +0000 Add support for the XDG Base Directory Specification Support configuration file placed in `$XDG_CONFIG_HOME/es-utils/config.yaml` Change: 0f799636f22bf8bff51c25a890f12341f6f7b9d7 Author: Brad Lhotsky Date : 2021-06-05 14:57:35 +0000 Fix warning in es_request() When the index is undefined, the code throws a warning. Change: fa084386a44bbda1a75ee961f8ea6228ab62fe0c Author: Brad Lhotsky Date : 2021-03-20 17:17:13 +0000 The GitHub::Meta plugin stopped working and needed configuration Change: c118b27b05d5e5e75b444b9537362d78e099176f Author: Brad Lhotsky Date : 2020-10-08 13:42:10 +0000 Fix warnings when we encounter an unallocated shard ---------------------------------------- version 7.8 at 2020-09-16 18:12:23 +0000 ---------------------------------------- Change: 7dbb50ff27914898417120f617def09279f8adc0 Author: Brad Lhotsky Date : 2020-09-16 11:12:23 +0000 Better handling of percentages For `--top` queries, be consistent about handling of the number of decimal places. Default to showing 3 digits to the right of the decimal, but allow a user to override it with `--precision`. Show the percentages in the summary when using `--days` greater than 1. Update the documentation. Change: e55ca5e32f64bc10c1def43c4766b93fe1430679 Author: Brad Lhotsky Date : 2020-09-04 13:00:37 +0000 Add percentages to es-search.pl aggs Whenever an aggregation is used, output the percentage from 0-1 with the doc count. Additionally, in sub aggs, calculate the percentage of the parent bucket. Change: 27a31a79da70aef08ec04586b9496160f2127007 Author: Brad Lhotsky Date : 2020-06-08 09:52:42 +0000 Remove the HOME paths from the lookup In the previous release, I added a check for the HOME environment variable, but I neglected to remove the originals from the search path. ---------------------------------------- version 7.7 at 2020-06-08 15:34:40 +0000 ---------------------------------------- Change: 5cbe34afdc85b490d2938aa9e37a39f9b34ae6d6 Author: Brad Lhotsky Date : 2020-06-08 08:34:40 +0000 Bugfix release * Sending "0" value query parameters did not work previously * Only add the ~/.es-utils.* to the config search path if HOME is set * Fix typo in the documentation ---------------------------------------- version 7.6 at 2020-03-27 22:43:58 +0000 ---------------------------------------- Change: 6cb7406e8cf4b112efe4fffd0a0138089d841597 Author: Brad Lhotsky Date : 2020-03-27 15:43:58 +0000 Release Version 7.6 * In `::Query`, the `fields` element was deprecated a long time ago, so overload it as a source filter. * `es-search.pl` * Ensure `--missing` and `--exists` can be specified more than once * Enable source filtering when the user specifies `--show` * Honor `--max-batch-size` when `--all` is specified Change: 8ef098984362da46348bec5bbb66de15f57c07a7 Author: Brad Lhotsky Date : 2020-03-09 17:20:26 +0000 Added es-index-blocks.pl to manage index blocks Change: 7804940e5562e5edc9e4472c7e9dcb3c9eeeb97e Author: Brad Lhotsky Date : 2019-12-31 11:05:36 +0000 Ensure error message isn't blank Change: 6743298fac9c15c55ea719e61eb63d5397e2f60e Author: Brad Lhotsky Date : 2019-12-31 11:00:24 +0000 Change aggregation ID and make debug output more useful. The debug output is now in a better format to paste to a Kibana instance for debugging. Change: cc0f7b846e85d6a8397dea24ff4102083990cd3e Author: Vitaly Shupak Date : 2019-12-09 18:03:10 +0000 use LWP::UserAgent request instead of simple_request simple_request does not handle authentication responses, which breaks in environments where ElasticSearch requires Kerberos authentication. ---------------------------------------- version 7.5 at 2019-11-21 21:39:18 +0000 ---------------------------------------- Change: eef552859755c3b446755232ef5cdab4fe39713f Author: Brad Lhotsky Date : 2019-11-21 13:39:18 +0000 Version 7.5 Release Change: fdccadaf74838e2876b5d370978ba780a8bbb819 Author: Brad Lhotsky Date : 2019-11-21 13:33:31 +0000 Include count in the --by ordering The default sort order for --by does not take into account the count of the sub buckets. This adds a secondary sort descending on the count of objects in each bucket. Change: a27a8b23f9bfd25259539dbae83a54ffc5a40cc7 Author: Brad Lhotsky Date : 2019-11-21 13:10:17 +0000 Make ::Query objects able to execute This was an oversight on my part. The `::Query` object now supports `->execute( $list_of_indexes )` directly. This makes working with the library easier as folks don't need to read docs on the `es_request()` function to figure out how to construct a search. Additionally, `scroll_id`'s are automatically extracted from the result set. Calls to `->scroll_results()` will allow scrolling without having to understand the mechanics of the scroll. ---------------------------------------- version 7.4 at 2019-10-20 13:24:50 +0000 ---------------------------------------- Change: be1145db74f2394ae9c4f2747332cc03d3f7ca07 Author: Brad Lhotsky Date : 2019-10-20 09:24:50 +0000 Ready v7.4 Release * Docs updated * Update tests to handle the new IP range query Change: feca6053d575b523277743a937a51cb226781d66 Author: Brad Lhotsky Date : 2019-10-20 09:03:07 +0000 Add Support for --or and IPv6 * Added support to change the default token joiner to OR to es-search.pl * Added support for IPv6 to the ::QueryString::IP expander * Converted ::QueryString::IP to use a range query Change: 17e80653b2a90e665e59c545eda7776b2fae7c90 Author: Brad Lhotsky Date : 2019-10-19 10:46:48 +0000 Update the examples/es-utils.yml file for new options This file had gotten outdated. Change: 31c3caf10224615723e8036eae28ac7789a1824d Author: Brad Lhotsky Date : 2019-10-09 20:23:57 +0000 Use a better name for fields metadata Change: f6369b88f311751908dec2addad33d999a452c75 Author: Brad Lhotsky Date : 2019-10-09 20:19:38 +0000 Some code cleanup for es-search.pl Change: 29bbf1415b2be39cf79d0281608832ade50565a9 Author: Brad Lhotsky Date : 2019-10-09 20:18:57 +0000 Allow field metadata sharing with ::Query This will enable automatic detection of and working with nested fields in the near term. Possibly other features in the long term. ================================================= Plus 80 releases after 2019-09-25 00:00:00 +0000. =================================================