Fix PostgreSQL migration for ban DB refactor (#42609)
This didn't come up in testing. The ban hit foreign key was added before the data was migrated, so it didn't work.
Fix that.
Ban database refactor (#42495)
* Ban DB refactor seems to work at a basic level for PostgreSQL
* New ban creation API
Supports all the new functionality (multiple players/addresses/hwids/roles/rounds per ban).
* Make the migration irreversible
* Re-implement ban notifications
The server ID check is no longer done as admins may want to place bans spanning multiple rounds irrelevant of the source server.
* Fix some split query warnings
* Implement migration on SQLite
* More comments
* Remove required from ban reason
SS14.Admin changes would like this
* More missing AsSplitQuery() calls
* Fix missing ban type filter
* Fix old CreateServerBan API with permanent time
* Fix department and role ban commands with permanent time
* Re-add banhits navigation property
Dropped this on accident, SS14.Admin needs it.
* More ban API fixes.
* Don't fetch ban exemption info for role bans
Not relevant, reduces query performance
* Regenerate migrations
* Fix adminnotes command for players that never connected
Would blow up handling null player records. Not a new bug introduced by the refactor, but I ran into it.
* Great shame... I accidentally committed submodule update...
* Update GDPR scripts
* Fix sandbox violation
* Fix bans with duplicate info causing DB exceptions
Most notably happened with role bans, as multiple departments may include the same role.
* Ban DB refactor seems to work at a basic level for PostgreSQL
* New ban creation API
Supports all the new functionality (multiple players/addresses/hwids/roles/rounds per ban).
* Make the migration irreversible
* Re-implement ban notifications
The server ID check is no longer done as admins may want to place bans spanning multiple rounds irrelevant of the source server.
* Fix some split query warnings
* Implement migration on SQLite
* More comments
* Remove required from ban reason
SS14.Admin changes would like this
* More missing AsSplitQuery() calls
* Fix missing ban type filter
* Fix old CreateServerBan API with permanent time
* Fix department and role ban commands with permanent time
* Re-add banhits navigation property
Dropped this on accident, SS14.Admin needs it.
* More ban API fixes.
* Don't fetch ban exemption info for role bans
Not relevant, reduces query performance
* Regenerate migrations
* Fix adminnotes command for players that never connected
Would blow up handling null player records. Not a new bug introduced by the refactor, but I ran into it.
* Great shame... I accidentally committed submodule update...
* Update GDPR scripts
* Fix sandbox violation
* Fix bans with duplicate info causing DB exceptions
Most notably happened with role bans, as multiple departments may include the same role.
* initial visual nubody
* oops overlay
* im so pheeming rn
* conversion...
* tests
* comeback of the underwear
* oops eyes
* blabbl
* zeds
* yaml linted
* search and visible count constraints
* reordering
* preserve previously selected markings colors
* fix test
* some ui niceties
* ordering
* make DB changes backwards-compatible/downgrade-friendly
* fix things again
* fix migration
* vulpkanin markings limit increase
* wrapping
* code cleanup and more code cleanup and more code cleanup and more code cleanup and
* fix slop ports
* better sampling API
* make filter work + use the method i made for its intended purpose
* fix test fails real quick
* magic mirror cleanup, remove TODO
* don't 0-init the organ profile data
* remove deltastates
---------
Co-authored-by: Princess Cheeseballs <66055347+Pronana@users.noreply.github.com>
* Make ServerPackaging automatically get extra server assemblies
* Make the switch
* Use Content.Server.deps.json instead
* Remove debug
* Rewrite
Now recursively fetches dependencies from Content.Server
Only copies dependencies not covered by Robust
This removes the need to manually specify most of the dependencies, even the content ones!
Also look at runtime key properly to figure out the proper dll name.
This actually removes some assemblies that were duplicated between the main directory and assemblies (various Microsoft.Extensions stuff)
* Fix test compile errors when updating dependencies
Ran across this while updating dependencies on the RT .NET 10 update. Should be fine to merge immediately.
* More .NET 10 prep
* Convert to SLNX
Hell yeah
* slnx now has size-2 indents
* Update SLNX with new RT system
* Remove reference to RT test in toolshed test
* Remove accidental usage of transitive RT dependencies
* Move Robust project references to RobustApi
* Update solution file
* Fix warnings in pow3r
* Fix nullable warnings in integration tests
idk where these came from
* gitignore binlog files
* Fix transitive dependency warnings in Content.Benchmarks
* Update slnx
* Okay, the Robust API thing didn't pan out. New plan.
It apparently broke clean builds, as the dependencies aren't in the project asset list or something anymore. I tried to fix this, but it seems impossible to do without relying on .NET SDK internals, as there's no point in the NuGet graph walk process that seems cleanly extensible.
Instead let's just do the much dumber thing: a bunch of .props files for content to import. Hooray!
This also means that I have to go through and *explicitly* disable transitive dependencies everywhere in RT. This thankfully isn't too hard.
* Update RT to 269.0.0
* One last solution update
* Fix more data definition issues
* Update RT to 269.0.1
* Fix it again
---------
Co-authored-by: DrSmugleaf <drsmugleaf@gmail.com>
* Persist construction menu favorites to player profile
* Use `ProtoId`s for construction favorites
* Validate construction favorites updates from the client
* Actually await the async database call
* Add loadout names
Did it for AI, breaking change for pgsql + migrations in general. Nothing atm uses it.
* the box
* Spawning cherry pick
* Fix nit
* revert
* Final cleanup
* Real
* Name UI fix
* Migrations
* a
* Review
* Re-run migrations
---------
Co-authored-by: Pieter-Jan Briers <pieterjan.briers+git@gmail.com>
This should be the primary changes for the future-proof "Modern HWID" system implemented into Robust and the auth server.
HWIDs in the database have been given an additional column representing their version, legacy or modern. This is implemented via an EF Core owned entity. By manually setting the column name of the main value column, we can keep DB compatibility and the migration is just adding some type columns.
This new HWID type has to be plumbed through everywhere, resulting in some breaking changes for the DB layer and such.
New bans and player records are placed with the new modern HWID. Old bans are still checked against legacy HWIDs.
Modern HWIDs are presented with a "V2-" prefix to admins, to allow distinguishing them. This is also integrated into the parsing logic for placing new bans.
There's also some code cleanup to reduce copy pasting around the place from my changes.
Requires latest engine to support ImmutableArray<byte> in NetSerializer.
Fixes formatting of owned entity type property names. These are normally named "FooBar_Baz" by EF Core, but the snake case thing was turning them into "foo_bar__baz". The double underscore is now fixed.
We don't *yet* have any EF Core owned entity in use, but I am planning to add one. I don't know if downstreams are using any so this should still be marked as a breaking change.
Also fixed it creating and dropping a Compiled Regex instance for every name, the regex is now cached (and pregenerated).
* Beginnings of making the breadmemes jobs easier
* stuff
* stuff pt. 2
* Stuff pt.3
* Stuff I forgot last time
* Basic whitelist
Only people that are added to the whitelist with the addwhitelist command will be able to join. I call this the "legacy" whitelist
* Remove always deny condition in favor of just breaking if playtime check fails
* Change default whitelist
Default whitelist is now the "legacy" whitelist.
* localization
* Admin check
* minor spelling change
* Fix build
* Whitelist message
* Fix vars not being datafield and spelling mistakes
* Minor spelling mistake
* Change config for salamander
* Reviews and stuff
* Add summaries
* Fix whitelists
* Forgot to add a datafield
* Fixing stuff I guess
* Reuse admin remarks to reduce load when connecting.
* Update log messages to be verbose instead of debug
* Reviews
* whoops
* Explain a bit more how whitelist checking works
* Apply CE's review
* Append Membership to Blacklist and Whitelist conditions
* Fix review comments
* Uncapitalize playerConnectionWhitelist, add to ignored client prototypes
* Make note count field work
* Fix cvar for thingy
---------
Co-authored-by: Pieter-Jan Briers <pieterjan.briers+git@gmail.com>
* Start work on PostgresNotificationManager
Implement initial version of init and listening code
* Finish implementing PostgresNotificationManager
Implement ban insert trigger
* Implement ignoring notifications if the ban was from the same server
* Address reviews
* Fixes and refactorings
Fix typo in migration SQL
Pull new code in BanManager out into its own partial file.
Unify logic to kick somebody with that when a new ban is placed directly on the server.
New bans are now checked against all parameters (IP, HWID) instead of just user ID.
Extracted SQLite ban matching code into a new class so that it can mostly be re-used by the ban notification code. No copy-paste here.
Database notifications are now not implicitly sent to the main thread, this means basic checks will happen in the thread pool beforehand.
Bans without user ID are now sent to servers. Bans are rate limited to avoid undue work from mass ban imports, beyond the rate limit they are dropped.
Improved error handling and logging for the whole system.
Matching bans against connected players requires knowing their ban exemption flags. These are now cached when the player connects.
ServerBanDef now has exemption flags, again to allow matching full ban details for ban notifications.
Made database notifications a proper struct type to reduce copy pasting a tuple.
Remove copy pasted connection string building code by just... passing the string into the constructor.
Add lock around _notificationHandlers just in case.
Fixed postgres connection wait not being called in a loop and therefore spamming LISTEN commands for every received notification.
Added more error handling and logging to notification listener.
Removed some copy pasting from SQLite database layer too while I was at it because god forbid we expect anybody else to do all the work in this project.
Sorry Julian
---------
Co-authored-by: Pieter-Jan Briers <pieterjan.briers+git@gmail.com>
Zekins3366