feat(privacy): reconnect the message export
The exporter has worked since v1.4.8. The form that drives it went out with the old settings window in May, which left PRIVACY.md promising an access request the plugin had no way to answer. New section in the data and privacy tab: time range, sender substring, channel groups, format, and a save dialog. Form state lives in the tab, not the config -- a filter describes one action, and a stale "last 7 days, sender Mira" reappearing weeks later is a worse start than an empty form. StreamForExport now takes a caller-owned connection. The reader stays open for as long as the file is written, seconds to minutes on a large history, and chat keeps arriving throughout -- so the primary connection would be read here and written by UpsertMessage at once, and SqliteConnection is not thread-safe. Holding the read lock instead would trade that for freezing the game. ChannelGroups lifts the eight groups out of the deleted tab and finishes them: 37 of 89 channels belonged to no group and were therefore unreachable in the UI. Game Master channels follow ChatTypeExt.Parent(), so GmTell sits with the other tells rather than under system traffic -- an access request that quietly drops part of what it promises is the dangerous kind of gap. Also here: - OpenSecondaryConnection disposes on a failing pragma. Open can succeed and journal_mode=WAL still time out, and with Pooling=false the connection then survives until a finalizer reaches it. Affects the full-text rebuild worker too. - StreamForExport builds its logger before the reader, so a throwing CreateLogger cannot leave a reader nobody owns. - The export thread takes the gate itself instead of the caller taking it first. Acquiring before Start would strand the gate for the session if thread creation failed, and the gate also holds back the sweep. - Notifications are skipped once teardown has started. The thread has no cancellation path and finishing the file is right, but reporting it to a plugin that is gone is not. - Transient widget rows that return their value instead of saving it. Writing the config file on every keystroke of a sender filter would be both pointless and slow. - Five translated keys for "another database operation is running", in all 25 languages. Two of the four operation names have no trigger yet; they arrive with the cleanup and maintenance sections.
This commit is contained in:
+50
-34
@@ -710,9 +710,21 @@ internal class MessageStore : IDisposable
|
||||
internal SqliteConnection OpenSecondaryConnection()
|
||||
{
|
||||
var conn = new SqliteConnection(BuildConnectionString(DbPath));
|
||||
conn.Open();
|
||||
ApplyPragmas(conn);
|
||||
return conn;
|
||||
try
|
||||
{
|
||||
conn.Open();
|
||||
ApplyPragmas(conn);
|
||||
return conn;
|
||||
}
|
||||
catch
|
||||
{
|
||||
// Open can succeed and ApplyPragmas still throw: journal_mode=WAL
|
||||
// needs a lock and gives up after DefaultTimeout. Without this the
|
||||
// connection is neither returned nor closed, and with Pooling=false
|
||||
// it survives until a finalizer gets to it.
|
||||
conn.Dispose();
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
// Worker-only mutator. The bulk-insert worker is the single legitimate
|
||||
@@ -976,53 +988,57 @@ internal class MessageStore : IDisposable
|
||||
|
||||
// Streams messages for export, sorted ascending by Date, excluding soft-deleted rows.
|
||||
// Optional filters: chatTypes, from/to inclusive date range.
|
||||
// Caller is responsible for disposing the enumerator.
|
||||
// Lock caveat: lock guards command setup and ExecuteReader; the returned
|
||||
// MessageEnumerator is iterated lazily by the caller outside the lock.
|
||||
// Acceptable for v1.4.8 -- DbViewer iterates on its filter-worker Task and
|
||||
// any clash with UpsertMessage on the primary Connection is rare and
|
||||
// serialised by SQLite's own connection-level lock. v1.5.x DI cycle should
|
||||
// address this with a snapshot-to-list or connection pool.
|
||||
// Caller is responsible for disposing the enumerator and the connection.
|
||||
//
|
||||
// Takes a caller-owned connection from OpenSecondaryConnection rather than
|
||||
// using the primary one, and therefore takes no lock. The reader stays open
|
||||
// for as long as the export writes, which is seconds to minutes on a large
|
||||
// history, and chat keeps arriving throughout -- so the primary connection
|
||||
// would be read here and written by UpsertMessage at the same time, and
|
||||
// SqliteConnection is not thread-safe. Holding _readLock for the whole
|
||||
// export would trade that for freezing the game instead.
|
||||
//
|
||||
// WAL gives readers their own snapshot, so a live write cannot tear the
|
||||
// export mid-file either.
|
||||
internal MessageEnumerator StreamForExport(
|
||||
SqliteConnection conn,
|
||||
IReadOnlyCollection<int>? chatTypes,
|
||||
DateTimeOffset? from,
|
||||
DateTimeOffset? to
|
||||
)
|
||||
{
|
||||
lock (_readLock)
|
||||
{
|
||||
var cmd = Connection.CreateCommand();
|
||||
var cmd = conn.CreateCommand();
|
||||
|
||||
var clauses = new List<string> { "deleted = false" };
|
||||
if (chatTypes is { Count: > 0 })
|
||||
clauses.Add($"ChatType IN ({BindIntList(cmd, "exct", chatTypes)})");
|
||||
if (from is not null)
|
||||
clauses.Add("Date >= $From");
|
||||
if (to is not null)
|
||||
clauses.Add("Date <= $To");
|
||||
var clauses = new List<string> { "deleted = false" };
|
||||
if (chatTypes is { Count: > 0 })
|
||||
clauses.Add($"ChatType IN ({BindIntList(cmd, "exct", chatTypes)})");
|
||||
if (from is not null)
|
||||
clauses.Add("Date >= $From");
|
||||
if (to is not null)
|
||||
clauses.Add("Date <= $To");
|
||||
|
||||
cmd.CommandText =
|
||||
@"
|
||||
cmd.CommandText =
|
||||
@"
|
||||
SELECT
|
||||
Id, Receiver, ContentId, Date, ChatType, SourceKind, TargetKind,
|
||||
Sender, Content, SenderSource, ContentSource, ExtraChatChannel
|
||||
FROM messages
|
||||
WHERE "
|
||||
+ string.Join(" AND ", clauses)
|
||||
+ @"
|
||||
+ string.Join(" AND ", clauses)
|
||||
+ @"
|
||||
ORDER BY Date ASC;";
|
||||
cmd.CommandTimeout = 600;
|
||||
cmd.CommandTimeout = 600;
|
||||
|
||||
if (from is not null)
|
||||
cmd.Parameters.AddWithValue("$From", from.Value.ToUnixTimeMilliseconds());
|
||||
if (to is not null)
|
||||
cmd.Parameters.AddWithValue("$To", to.Value.ToUnixTimeMilliseconds());
|
||||
if (from is not null)
|
||||
cmd.Parameters.AddWithValue("$From", from.Value.ToUnixTimeMilliseconds());
|
||||
if (to is not null)
|
||||
cmd.Parameters.AddWithValue("$To", to.Value.ToUnixTimeMilliseconds());
|
||||
|
||||
return new MessageEnumerator(
|
||||
cmd.ExecuteReader(),
|
||||
_loggerFactory.CreateLogger<MessageEnumerator>()
|
||||
);
|
||||
}
|
||||
// Logger first: an argument list evaluates left to right, so a throwing
|
||||
// CreateLogger -- which is what a disposed host gives you -- would leave
|
||||
// an open reader that no MessageEnumerator owns.
|
||||
var logger = _loggerFactory.CreateLogger<MessageEnumerator>();
|
||||
return new MessageEnumerator(cmd.ExecuteReader(), logger);
|
||||
}
|
||||
|
||||
// Returns the most recent messages, oldest-first.
|
||||
|
||||
Reference in New Issue
Block a user