Revision history for Database-Abstraction 0.39 Sun Aug 23 09:12:54 PM EDT 2026 [ Bug Fixes ] - Bump minimum Log::Abstraction - Fix https://github.com/nigelhorne/Database-Abstraction/issues/13 [ Tests ] - t/path.t: control-flow path-coverage tests for all key internal helpers (_like_match, _match_criterion, _has_complex_criteria, _build_where, _build_where_conditions, _build_joins, _is_local_host, _fixate via public-API data integrity checks, _quote_identifier, _is_deep_db, _open_table, AUTOLOAD); 102 assertions covering every uniquely identifiable branch - t/transaction.t: transaction-flow and resource-lifecycle tests covering object lazy-open→query→DESTROY sequencing, query idempotency across all four select methods, multi-instance isolation (independent data refs, Data::Reuse stale-address safety), CHI cache MISS→SET→HIT lifecycle, gzip temp-file born/reused/unlinked on DESTROY, and query-builder chain idempotency; 31 assertions - t/path.t P8-2, t/data-flow.t DF3: replaced hash-key-lock die assertions with portable data-integrity checks; hash-key locking via Data::Reuse / Hash::Util is not guaranteed to throw on all platforms — some CPAN testers return undef instead; the fix tests that fixated data is a HASH with expected keys (P8-2) and that the same row is stable across repeated access (DF3), which guards the actual invariant without depending on the locking mechanism Fixes https://www.cpantesters.org/cpan/report/83f9464a-9e6f-11f1-b690-edda6d8775ea 0.38 Sat Aug 22 15:21:37 EDT 2026 [ Enhancements ] - HTML URL backend: new url => 'https://...' constructor parameter fetches a remote HTML page with LWP::UserAgent, extracts a element via HTML::TableExtract, and slurps it into the same in-memory data structure used by CSV/XML slurp; the first row of the table is treated as column headers; html_table_index (default 0) selects which table on the page to use; directory is not required when url is given; both LWP::UserAgent and HTML::TableExtract are loaded lazily (only when url is given); the URL is validated in new() against /\Ahttps?:\/\// at construction time (ftp://, file://, javascript: and all other schemes are rejected); all in-memory fast-paths (selectall_arrayref, selectall_array, fetchrow_hashref, count, AUTOLOAD, columns, schema) work without modification; type is set to 'HTML'; no DBI handle is created; t/html.t covers keyed and no_entry modes, html_table_index selection, AUTOLOAD lookup, repeated-call fetch-once guard, and all error paths 0.37 Mon Jul 27 16:58:13 EDT 2026 [ Enhancements ] - Remote file support: new host => 'server' (or 'user@host') constructor parameter; when present, all candidate file extensions are fetched from the remote machine via File::Slurp::Remote (SSH/SCP) into a local File::Temp directory before the existing extension-based probe runs; all backends (CSV, PSV, XML, SQLite, DBM::Deep) work transparently; File::Temp::Dir object stored in _remote_tmpdir is auto-cleaned in DESTROY; host name validated in new() against a strict identifier regex (prevents shell injection); the local -d directory check is skipped when host is given (directory is a remote path); File::Slurp::Remote is loaded lazily (only when host is given) - Local-host short-circuit: when host => 'localhost', '127.0.0.1', '::1', or the current machine's hostname (short or FQDN) is given, _open() reads the directory directly without loading or calling File::Slurp::Remote; implemented by _is_local_host() which strips optional user@ prefix then matches loopback literals and Sys::Hostname::hostname() (both short and FQDN forms); new() restores the local -d directory check for local hosts; host validation regex extended to allow ':' so IPv6 addresses (::1, etc.) are accepted - DBM::Deep backend: files with .dbm or .deep extension are detected (after SQLite, before BerkeleyDB) and slurped into the in-memory hash at open time via DBM::Deep->new(read_only=>1); all existing slurp fast-paths (selectall_arrayref, selectall_array, fetchrow_hashref, AUTOLOAD, columns, schema) work unchanged; type is set to 'Deep' - DBM::Deep magic-byte detection: _is_deep_db() checks the first 4 bytes for 'DPDB' (0x44 0x50 0x44 0x42, the standard DBM::Deep signature) or 'DPDP' (0x44 0x50 0x44 0x50); in _open() the .db candidate is probed for Deep magic before the BerkeleyDB check, so .db files that are actually DBM::Deep are correctly identified without requiring a .dbm or .deep extension - count() general in-memory scan: added a third fast-path branch that fires when $self->{'data'} is set AND there is no DBI handle (i.e. Deep backend); allows count(col => $val) without DBI; SQL-backed backends (CSV/XML/SQLite) still fall through to SQL so column-name validation continues to fire - Query builder (Query.pm) all()/first()/count(): added Deep alongside BerkeleyDB in the in-memory delegate path so that query()->where()->all() etc. work on Deep files without a DBI handle [ Security ] - new() clone path: validate 'id' argument before bless in the clone branch; previously the early return at line 540 bypassed the id validation block at lines 558-562, allowing $obj->new(id => "x); DROP TABLE t--") to produce an object whose $self->{'id'} was directly interpolated (unbound) into ORDER BY, COUNT(), and CSV-comment-filter SQL clauses; fix adds the same /^[a-zA-Z_][a-zA-Z0-9_]*$/ guard inside the clone branch before bless [ Bug fixes ] - t/{csv,csv_id,schema,locales,no_entry,bugs}.t: replaced use_ok() with compile-time use + pass() to prevent "Too late to run CHECK block" warnings from Sub::Private (used by Log::Abstraction); use_ok() wraps its argument in a runtime string eval, past the CHECK phase, which caused Test::NoWarnings failures in those six test files - Don't attempt to fixate empty data - _fixate(): new private wrapper around Data::Reuse::fixate() that suppresses the spurious "Use of uninitialized value in hash slice" warning emitted by Data::Alias's XS hash-aliasing code on older Perl versions when a source hashref contains undef column values (NULL database fields); Data::Reuse still fixates correctly — the warning is a false positive from Data::Alias not fully initialising key SVs in the older internal Perl hash representation; all four fixate call-sites in Abstraction.pm now go through _fixate(); confirmed reproducible with Data::Alias 1.30 on Perl < 5.34 - t/bugs.t BUG10: regression guard for the above warning across all three fixate code-paths (slurp, selectall_arrayref DBI, selectall_array DBI) - _fixate(): move Data::Reuse::forget() inside the wrapper so it covers all four fixate call-sites including the slurp path; the slurp fixate was previously exempt on the grounds that slurped hashrefs are long-lived and don't create stale entries themselves, but it was still vulnerable to stale entries left by earlier objects whose hashrefs were freed after going out of scope — the memory allocator can reuse those addresses for new slurp hashrefs, causing fixate() to alias them to wrong canonicals (root cause of an intermittent t/unit.t failure where $db->number('one') occasionally returned 2 instead of 1); the explicit Data::Reuse::forget() calls that previously preceded _fixate() in selectall_arrayref and selectall_array are now redundant and have been removed 0.36 Thu Jul 9 21:38:49 EDT 2026 [ Bug fixes ] - Data::Reuse stale-address hazard: selectall_arrayref() and selectall_array() now call Data::Reuse::forget() immediately before fixate() to clear stale address→canonical mappings that could cause one row's data to be silently substituted for another's on repeated DBI calls (intermittent wrong-row returns in join queries) - Windows-portable file detection: replaced File::pfopen (which splits the path on ':' and breaks Windows drive letters) with File::Spec->catfile + -r checks in _open_table() - t/locales.t: trim_loc() regex now uses /s flag so it correctly strips multi-line Carp::confess stack traces, not just single-line croak output [ Security ] - Input validation / SQL injection guards: id column name (new()), dbname and filename (_open()), and join table name (_build_joins()) are now validated against strict identifier regexes; invalid values croak with a descriptive message before any SQL is constructed - _like_match(): replaced regex-based LIKE translation (%->.*) with an O(m*n) dynamic-programming matcher in slurp mode, eliminating catastrophic backtracking on patterns such as %a%a%a%a%b [ Enhancements ] - Prepared statement caching: all DBI prepare() calls replaced with prepare_cached() for statement-handle reuse across queries - Schema introspection: new columns() and schema() methods; SQLite uses PRAGMA table_info, other DBI drivers use column_info, slurp mode inspects the first row; results are cached after the first call - DSN portability: new() now accepts dsn/username/password parameters, bypassing file detection entirely; dialect is auto-detected from the DSN prefix (sqlite/postgres/mysql); SQLite performance PRAGMAs are only applied when connecting via a sqlite DSN - Operator-hash query criteria: all select methods now accept hashref values for comparison operators (> < >= <= != -like -not_like -in -not_in -between) and logical groupings (-or -and); backed by new private helpers _build_where / _build_where_conditions / _match_criterion / _has_complex_criteria - Automatic joins: selectall_arrayref(), selectall_array(), and fetchrow_hashref() accept a join parameter (hashref or arrayref of hashrefs with keys table/on/type); backed by _build_joins(); valid JOIN types are INNER LEFT RIGHT FULL CROSS (default INNER) - Chained query builder: new query() method returns a Database::Abstraction::Query object supporting fluent method chaining: ->select()->where()->join()->order_by()->limit()->offset()->all()/ first()/count() - Use Class::Abstract to ensure it's not instantiated directly - DBD::SQLite::Constants and Data::Dumper are now loaded lazily (only when the relevant code path is actually exercised) 0.35 Mon Jun 8 08:25:03 EDT 2026 [ Bug fixes ] - Pass f_dir on open - Canonicalize directory path via Cwd::abs_path to fix DBD::File safety check rejecting paths containing '..' [ Enhancements ] Updated to the latest test dashboard 0.34 Mon Jan 19 21:20:49 EST 2026 Fix logger with array test when a configuration file has been set up Allow execute() to take an argument(s) 0.33 Sat Oct 18 18:35:46 EDT 2025 Using undef to searches sets IS NULL Latest testing dashboard 0.32 Tue Sep 16 08:26:41 EDT 2025 Fix fast track return in selectall_arrayref 0.31 Mon Sep 15 20:15:00 EDT 2025 Call sth->finish() when needed selectall_arrayref was setting an extra level of reference Added the test dashboard 0.30 Sun Aug 24 15:11:14 EDT 2025 slurp mode - remove a data copy selectall_hashref now returns undef if there are no matches rewrote selectall_hashref to not just call selectall_hash, that saves data copying Imply 'entry' argument when possible Added the _fatal method selectall_hashref name is misleading, will be selectall_arrayref selectall_hash name is misleading, will be selectall_array Fix potential memory leak issues 0.29 Mon Aug 18 21:42:25 EDT 2025 Moved the test data from data to t/data Fixed wildcard usage determination Optimise SQLite Pragmas Removed data copy when slurping XML with no_entry set Selectall_hashref - remove data copy Reduce chance of cache key name clash 0.28 Wed Aug 6 19:46:59 EDT 2025 Added the import method Fix determination of BerkeleyDB Added BerkeleyDB test Increase SQLite timeout Added the count() method AUTOLOAD - sanity check on the column name 0.27 Sat Jun 14 20:56:27 EDT 2025 Use Object::Configure rather than Class::Debug 0.26 Tue Apr 15 16:42:48 EDT 2025 Added the 'no_fixate' option POC - fixate slurped data 0.25 Sun Apr 13 08:00:22 EDT 2025 Fix auto_load calling unknown routine isFalse Removed _info and _notice as they're not used More extensive use of Params::Get to clean the code Added sanity test Allow subclasses to set max_slurp_size on open() Improved slurping of XML files - still needs more work Use Data::Reuse, but note RT#100461 0.24 Wed Mar 26 11:46:57 EDT 2025 Added basic support for BerkeleyDBs (files ending in .db) If config_file has a section for this class, use that 0.23 Tue Mar 25 08:06:33 EDT 2025 Not all loggers (e.g. Log::Any) can cope with a reference to an ARRAY 0.22 Fri Mar 14 08:02:39 EDT 2025 Allow the AUTOLOAD feature to be disabled Added the auto_load parameter to disable AUTOLOAD Force _open/_log to be a private/protected method Use Params::Get and Log::Abstraction Added config_file argument to new() 0.21 Tue Feb 11 07:45:15 EST 2025 Fix "Not an ARRAY reference" 0.20 Thu Feb 6 20:29:09 EST 2025 Fix the manifest 0.19 Thu Feb 6 20:19:07 EST 2025 Fix selectall_hashref() in slurp mode when no_entry is set 0.18 Thu Feb 6 07:26:02 EST 2025 Cache hits in list context returns a ref to the array, rather than the array itself 0.17 Tue Feb 4 15:32:43 EST 2025 Support expires_in for compatability with CHI Added cache support to AUTOLOAD Allow the logger to be a filename When the logger is a reference to code, give more information Added the 'filename' property Don't attempt to slurp files when col_names is set 0.16 Sun Jan 12 12:15:01 EST 2025 fetchrow_hashref() - when no_entry is not set allow just one argument to be given: the entry value 0.15 Thu Jan 9 19:43:43 EST 2025 Improved handling of empty values in slurp mode AUTOLOAD: error on invalid column name in slurp mode 0.14 Fri Jan 3 11:17:11 EST 2025 Added the dbname argument 0.13 Sun Dec 22 13:30:13 EST 2024 Refactor logger helper routines Fix "Bad table or column name: 'entryIS NOT NULL' has chars not alphanumeric or underscore" Added t/30-basics.t 0.12 Wed Dec 4 08:19:09 EST 2024 Move the synopsis document around for clarity Allow the logger to be a ref to code 0.11 Thu Nov 7 10:44:24 EST 2024 Added id parameter Slurp small XML files using XML::Simple Fix uninitialised error when slurp_file doesn't exist Use features of File::pfopen version 0.03 0.10 Thu Sep 19 08:17:55 EDT 2024 Added t/init.t The constructor now honours 'sep_char' Test no_entry files Added CircleCI and Codecov 0.09 Mon Jun 24 20:42:48 EDT 2024 Added t/chi.t selectall_hash: speed up slurp mode when 'entry' is one of the parameters execute: add the table name to the query 0.08 Mon May 20 08:43:08 EDT 2024 Add unique as a synonym for distinct Allow max_slurp_size to be tweaked Fix CSV search on a column other than the keyed column Added t/error.t Catch undefined arguments in fetchrow_hashref 0.07 Fri Feb 23 08:02:24 EST 2024 selectall_hash: Fixed "Not an ARRAY reference" 0.06 Tue Feb 20 16:55:33 EST 2024 Slurped data - improve keying fetchrow_hashref now uses slurped data on some optimised places fetchrow_hashref: fixed uninitialized variable in logger fetchrow_hashref: more verbose debugging on match AUTOLOAD: usage error if entry isn't a column AUTOLOAD: fix problem when no args are given on slurped data 0.05 Fri Feb 9 11:01:00 EST 2024 Fix https://www.cpantesters.org/cpan/report/f07e90f2-c264-11ee-bb2d-64236e8775ea init() will now return the current defaults Ensure that the default cache_duration is 1 hour 0.04 Fri Feb 2 13:14:18 EST 2024 init() can now be called more than once and will retain the old values Added XML test Restrict the size of the CSV file that are slurped to 16K 0.03 Thu Feb 1 13:48:01 EST 2024 Complain if the directory argument isn't a directory Put the values from init() into a hash Remove Database::Abstraction::Error - it's an artifact It was a good idea, but Error doesn't play well with 'use lib' 0.02 Tue 30 Jan 11:07:12 EST 2024 Simpler package name used for testing Fix the Error objection being thrown 0.01 Mon Jan 29 14:02:37 EST 2024 First draft