Commit Graph

159 Commits

Author SHA1 Message Date
Azareal
ad4763a883 use string builder pool for mysql query strings
get more mysql queries to use string builders
use consts instead of vars for growth hint variables
2020-04-21 09:39:35 +10:00
Azareal
a47a0318a7 increase the number of agents on the ua chart
optimise mfaVerifySession
reduce boilerplate for populating the menu items
shorten ua phrases for semrush and aspiegel
rename account_username_updated phrase to account_name_updated
2020-03-24 12:07:30 +10:00
Azareal
b04d77d7b6 optimise topic lists by caching common qcounts for getList stmts
recalc forum topic counts
reduce thaw period to 3
cache top 8 forums instead of 5
optimise GetListByForum with zero topics
add Each method to ForumStore
2020-03-09 13:51:44 +10:00
Azareal
f5118d1b11 fix possible linter screw-up in buildFlexiWhere 2020-03-05 11:01:23 +10:00
Azareal
1bee9a35f2 use string builder in buildOrderby 2020-03-05 10:40:40 +10:00
Azareal
c31ca03a0d use string builder in buildFlexiWhere
add string builder growth hints
track favicon perf
2020-03-05 09:56:45 +10:00
Azareal
ca8411a519 experimental route perf pane
add 1.14 to travis test list
more qutils tests

add panel_menu_stats_routes_perf phrase
2020-02-28 14:52:45 +10:00
Azareal
ea1037bd63 track favicon stats
experiment with tracking average route performance
temporary error route stub
optimise dumprequest
add DisableAnalytics config setting
fix double hyphens in slugs being mistaken for sql injection
more querygen tests

You wil need to run the updater / patcher for this commit.
2020-02-26 20:34:38 +10:00
Azareal
fb942fd100 optimise route counter
allow for multiple keys in mysql AddKey
add SimpleBulkInsert method
add BulkInsert acc builder
add DateOlderThanQ AccSelectBuilder method
2020-02-24 21:15:17 +10:00
Azareal
3d1fd807d3 add RemoveIndex adapter method
add AddKey installer method
2020-02-21 09:55:45 +10:00
Azareal
6935637867 Cascade delete attachments properly.
Cascade delete replied to topic events for replies properly.
Cascade delete likes on topic posts properly.
Cascade delete replies and their children properly.
Recalculate user stats properly when items are deleted.
Users can now unlike topic opening posts.
Add a recalculator to fix abnormalities across upgrades.
Try fixing a last_ip daily update bug.

Add Existable interface.
Add Delete method to LikeStore.
Add Each, Exists, Create, CountUser, CountMegaUser and CountBigUser methods to ReplyStore.
Add CountUser, CountMegaUser, CountBigUser methods to TopicStore.
Add Each method to UserStore.
Add Add, Delete and DeleteResource methods to SubscriptionStore.
Add Delete, DeleteByParams, DeleteByParamsExtra and AidsByParamsExtra methods to ActivityStream.
Add Exists method to ProfileReplyStore.

Add DropColumn, RenameColumn and ChangeColumn to the database adapters.

Shorten ipaddress column names to ip.
- topics table.
- replies table
- users_replies table.
- polls_votes table.
Add extra column to activity_stream table.
Fix an issue upgrading sites to MariaDB 10.3 from older versions of Gosora. Please report any other issues you find.

You need to run the updater / patcher for this commit.
2020-01-31 17:22:08 +10:00
Azareal
e4cfe610f6 Update last reply on topics properly upon reply delete. 2020-01-14 20:29:23 +10:00
Azareal
69a2430e5a Add option to delete all of a user's posts.
Add limited dynanicism to langf.
Eliminate a few queries when updating user stats.
Minor clean-up.

Namespace the profile language strings.
2020-01-14 15:07:00 +10:00
Azareal
f789a8498e Add PollIPCutoff configuration setting.
Add tests for qgen processWhere and qgen mysql buildWhere.

Shorten some things.
2020-01-02 15:28:36 +10:00
Azareal
53212f3022 Optimise buildWhere in the MySQL Adapter. 2020-01-01 18:53:48 +10:00
Azareal
08864edd6c Optimise DBWhere methods.
Optimise DBSetter methods.
2020-01-01 14:15:11 +10:00
Azareal
0a0a2ea2b0 Optimise insert field identifiers.
Optimise db where parse column.
2020-01-01 11:26:15 +10:00
Azareal
35ddc89009 Optimise the database layer.
Refactor database adapters.
Experimental last ip cutoff.

More parser test cases.
2020-01-01 07:57:54 +10:00
Azareal
c928c84c95 Reduce boilerplate in the currently defunct mssql.go and pgsql.go 2019-10-31 17:25:56 +10:00
Azareal
033f4624c8 Experiment with the accumulator.
Less boilerplate.
2019-10-29 11:58:04 +10:00
Azareal
2e28ae39f3 Add posts as a criteria for group promotions.
Load the posts column when loading users.
Make DefaultUserStore.Reload a wrapper around DefaultUserStore.BypassGet.
Use the OO style query builder more in common/user.go
GroupPromotions is now bound to an interface rather than the concrete type.
Fix some issues in the GroupPromotionStore interface.
Improve error handling for table creation and reduce boilerplate.
Shorten some more things.

Add panel_group_promotions_posts phrase.

You will need to run the patcher / updater for this commit.
2019-10-07 08:20:37 +10:00
Azareal
ddb917761f Show the names of the participants on the convo list instead of the placeholder text.
Shorten some things to save a bit of memory and to standardise things.
2019-08-28 16:47:54 +10:00
Azareal
4d8c97812d More conversations work.
Add support for AS in columns for SimpleInnerJoin.
Add a referrer policy to improve privacy a little.
Shorten /static/ to /s/ since it comes up so much.
Remove some obsolete code.
Shorten some variable names.
Reduce the amount of boilerplate in the patcher.

Added the RefNoTrack and RefNoRef privacy config settings.

You may need to run the updater / patcher for this commit.
2019-08-14 20:39:04 +10:00
Azareal
ef4a88c43f Fix an index related bug in the patcher.
Fix AS in columns in left joins.
Remove a redundant AS in simple count.
Remove a couple of unused session parameters.
Shorten a few session bits.
2019-07-29 11:31:19 +10:00
Azareal
22cb2701b2 Shorten a couple of attachment parameters.
Rename a mislabelled variable.
Prealloc in DefaultUserStore.BulkGetMap.
Prealloc in DefaultTopicStore.BulkGetMap.
Shorten the names of some temps.
Shorten a mssql querystring.
2019-07-25 17:00:49 +10:00
Azareal
12d733932b Optimise SimpleInsert. 2019-06-11 22:43:33 +10:00
Azareal
7bc2146088 Optimise SimpleUpdate. 2019-06-11 15:44:31 +10:00
Azareal
92250fdd8f Optimise ComplexSelect. 2019-06-11 15:23:25 +10:00
Azareal
ac0dd6d2cb The login and registration logs are pruned after a year now.
Add filter exclusions for the user agents analytics pane.
Shorten some of the variable names in qgen.

Add ComplexDelete to the query adapter.
Add DateCutoff and DateOlderThan to accDeleteBuilder.
2019-06-05 14:57:10 +10:00
Azareal
e76223224a Implemented periodic post IP purging for replies.
Update for MySQL now supports DateCutoff and DateOlderThan.
Added the DateOlderThan method to accUpdateBuilder.

Added the PostIPCutoff config setting.
2019-05-08 19:50:10 +10:00
Azareal
839df17de3 Experimenting with speeding up the installer.
Added support for foreign keys to the MySQL adapter.
activity_stream_matches now has a foreign key to help enforce referential integrity.
Added the AddForeignKey method to the database adapters.
Shortened a couple of API bits to ever slow slightly reduce the lengths of the strings.

Fixed a phrase group I missed for logged out users in init.js
Fixed a bug where deleting a topic would break the alert list when there is an alert event relating to it.

You will need to run the updater / patcher for this commit.
2019-05-06 14:04:00 +10:00
Azareal
414d9c4817 Tighter ratelimiting for password resets.
Eliminated a line in agents.go
Added the DateCutoff method to accCountBuilder.
Function columns should now work for ComplexSelect.
Added type=search to the Search and Filter Widget search box.

Began cleaning some prebuilder logic up.
Began work on the generic ratelimit interface.
2019-03-12 19:13:57 +10:00
Azareal
2296008655 Basic search now works for the Search & Filter Widget. ElasticSearch has been temporarily delayed, so I can push through this update.
Added the three month time range to the analytics panes.
Began work on adding new graphs to the analytics panes.
Began work on the ElasticSearch adapter for the search system.
Added the currently limited AddKey method to the database adapters.
Expanded upon the column parsing logic in the database adapters to ease the use of InsertSelects.

Added the BulkGet method to TopicCache.
Added the BulkGetMap method to TopicStore.
TopicStore methods should now properly retrieve lastReplyBy.
Added the panel_analytics_script template to de-dupe part of the analytics logic. We plan to tidy this up further, but for now, it'll suffice.
Added plugin_sendmail and plugin_hyperdrive to the continuous integration test list.
Tweaked the width and heights of the textareas for the Widget Editor.
Added the AddKey method to *qgen.builder
Fixed a bug where using the inline forum editor would crash Gosora and wouldn't set the preset permissions for that forum properly.
Added DotBot to the user agent analytics.
Invisibles should be better handled when they're encountered now in user agent strings.
Unknown language ISO Codes in headers now have the requests fully logged for debugging purposes.
Shortened some of the pointer receiver names.
Shortened some variable names.

Added the dotbot phrase.
Added the panel_statistics_time_range_three_months phrase.

Added gopkg.in/olivere/elastic.v6 as a dependency.

You will need to run the patcher or updater for this commit.
2019-02-23 16:29:19 +10:00
Azareal
d9acf27c5b The Search and Filter Widget is now partly implemented. Just Search to go in the basic implementation.
Added AJAX Pagination for the Topic List and Forum Page.
A new log file pair is now created every-time Gosora starts up.
Added proper per-theme template overrides.

Added EasyJSON to make JSON serialisation faster.
Moved a bit of boilerplate into paginator.html
Improved paginator.html with a richer template with first, last and symbols instead of text.
Phased out direct access to Templates.ExecuteTemplate across the software.
Fixed the Live Topic List so it should work again.
Added MicroAvatar to WsJSONUser for topic list JSON requests.
An instance of the plugin is now passed to plugin handlers rather than having the plugins manipulate the globals directly.
Added the pre_render_panel_forum_edit and pre_render_panel_forum_edit_perms hooks to replace pre_render_panel_edit_forum.
Renamed the pre_render_panel_edit_user hook to pre_render_panel_user_edit
Reduced the amount of noise from fsnotify.
Added RawPrepare() to qgen.Accumulator.
Added a temporary phrase whitelist to the phrase endpoint.
Moved the location of the zone data assignments in the topic list to reduce the chances of security issues in the future.
Changed the signature of routes/panel/renderTemplate() requiring some changes across the panel routes.
Removed bits of boilerplate in some of the panel routes with renderTemplate()
Added a BenchmarkTopicsGuestJSRouteParallelWithRouter benchmark.
Removed a fair bit of boilerplate for each page struct by generating a couple of interface casts for each template file instead.
Added the profile_comments_row_alt template.
Added the topics_quick_topic template to reuse part of the quick topic logic for both the topic list and forum page.
Tweaked the CSS for the Online Users Widget.
Tweaked the CSS for Widgets in every theme with a sidebar.
Refactored the template initialisers to hopefully reduce the amount of boilerplate and make things easier to maintain and follow.
Add genIntTmpl in the template initialiser file to reduce the amount of boilerplate needed for the fallback template bindings.

Removed the topics_head phrase.
Moved the paginator_ phrases into the paginator. namespace and renamed them accordingly.
Added the paginator.first_page phrase.
Added the paginator.first_page_aria phrase.
Added the paginator.last_page phrase.
Added the paginator.last_page_aria phrase.
Added the panel_forum_delete_are_you_sure phrase.

Fixed a data race in LogWarning()
2019-02-10 15:52:26 +10:00
Azareal
8f2f47e8aa Added the In-Progress Widget Manager UI.
Added the IsoCode field to phrase files.
Rewrote a good portion of the widget system logic.
Added some tests for the widget system.
Added the Online Users widget.
Added a few sealed incomplete widgets like the Search & Filter Widget.
Added the AllUsers method to WsHubImpl for Online Users. Please don't abuse it.

Added the optional *DBTableKey field to AddColumn.
Added the panel_analytics_time_range template to reduce the amount of duplication.
Failed registrations now show up in red in the registration logs for Nox.
Failed logins now show up in red in the login logs for Nox.
Added basic h2 CSS to the other themes.
Added .show_on_block_edit and .hide_on_block_edit to the other themes.
Updated contributing.
Updated a bunch of dates to 2019.
Replaced tblKey{} with nil where possible.
Switched out some &s for &s to reduce the number of possible bugs.
Fixed a bug with selector messages where the inspector would get really jittery due to unnecessary DOM updates.
Moved header.Zone and associated fields to the bottom of ViewTopic to reduce the chances of problems arising.
Added the ZoneData field to *Header.
Added IDs to the items in the forum list template.
Split the fetchPhrases function into the initPhrases and fetchPhrases functions in init.js
Added .colstack_sub_head.
Fixed the CSS in the menu list.
Removed an inline style from the simple topic like and unlike buttons.
Removed an inline style from the simple topic IP button.
Simplified the LoginRequired error handler.
Fixed a typo in the comment prior to DatabaseError()
Reduce the number of false leaves for WebSocket page transitions.
Added the error zone.
De-duped the logic in WsHubImpl.getUsers.
Fixed a potential widget security issue.

Added twenty new phrases.
Added the wid column to the widgets table.

You will need to run the patcher / updater for this commit.
2019-01-21 22:27:59 +10:00
Azareal
5db5bc0c7e Reply attachments can be managed too now.
Added eight database indices.
Fixed a bug where the second tick wouldn't fire.
Tweaked the .topic_forum in Nox by a pixel.
Replaced some _installer strings with blank strings for consistency with the builders.
Greatly reduced the number of allocations in the user agent parser.
Added ampersand entities in more attachment URLs to avoid accidental mangling.
.edit_source is now hidden for guests.
Guest noavatars are now pre-calculated to reduce the number of allocations.
Lazily initialised a couple of maps in ViewTopic to reduce the number of unnecessary allocations slightly.

Added the unsafe BytesToString function. Please don't use this, if you don't have to.
Added the AddIndex method to the adapter and associated components.
Added the /reply/attach/add/submit/ route.
Added the /reply/attach/remove/submit/ route.
Added the topic_alt_userinfo template.
Replaced Attachments.MiniTopicGet with MiniGetList.
Added Attachments.BulkMiniGetList.

Added a quick test for ReplyStore.Create.
Added BenchmarkPopulateTopicWithRouter.
Added BenchmarkTopicAdminFullPageRouteParallelWithRouter.
Added BenchmarkTopicGuestFullPageRouteParallelWithRouter.

You will need to run the updater or patcher for this commit.
2018-12-31 19:03:49 +10:00
Azareal
3465e4c08f You can now manage the attachments for an opening post by hitting edit.
The update system now uses the database as the source of truth for the last version rather than lastSchema.json
Refactored several structs and bits of code, so we can avoid allocations for contexts where we never use a relative time.
Clicking on the relative times on the topic list and the forum page should now take you to the post on the last page rather than just the last page.
Added the reltime template function.
Fixed some obsolete bits of code.
Fixed some spelling mistakes.
Fixed a bug where MaxBytesReader was capped at the maxFileSize rather than r.ContentLength.
All of the client side templates should work again now.
Shortened some statement names to save some horizontal space.
accUpdateBuilder and SimpleUpdate now use updatePrebuilder behind the scenes to simplify things.
Renamed selectItem to builder in AccSelectBuilder.
Added a Total() method to accCountBuilder to reduce the amount of boilerplate used for row count queries.
The "_builder" strings have been replaced with empty strings to help save memory, to make things slightly faster and to open the door to removing the query name in many contexts down the line.
Added the open_edit and close_edit client hooks.
Removed many query name checks.
Split the attachment logic into separate functions and de-duplicated it between replies and topics.
Improved the UI for editing topics in Nox.
Used type aliases to reduce the amount of boilerplate in tables.go and patches.go
Reduced the amount of boilerplate in the action post logic.
Eliminated a map and a slice in the topic page for users who haven't given any likes. E.g. Guests.
Fixed some long out-dated parts of the update instructions.
Updated the update instructions to remove mention of the obsolete lastSchema.json
Fixed a bug in init.js where /api/me was being loaded for guests.
Added the MiniTopicGet, GlobalCount and CountInTopic methods to AttachmentStore.
Added the MiniAttachment struct.
Split the mod floaters out into their own template to reduce duplication.
Removed a couple of redundant ParseForms.

Added the common.skipUntilIfExistsOrLine function.
Added the NotFoundJS and NotFoundJSQ functions.
Added the lastReplyID and attachCount columns to the topics table.
2018-12-27 15:42:41 +10:00
Azareal
bf851bd9fc We now use Go 1.11 modules. This should help with build times, deployment and development, although it does mean that the minimum requirement for Gosora has been bumped up from Go 1.10 to Go 1.11
Added support for dyntmpl to the template system.
The Account Dashboard now sort of uses dyntmpl, more work needed here.
Renamed the pre_render_view_topic hook to pre_render_topic.
Added the GetCurrentLangPack() function.
Added the alerts_no_new_alerts phrase.
Added the account_level_list phrase.

Refactored the route rename logic in the patcher to cut down on the amount of boilerplate.
Added more route renames to the patcher. You will need to run the patcher / updater in this commit.
2018-10-27 13:40:36 +10:00
Azareal
ad61b5d8b2 MySQL being stupid x.x - #3 2018-10-07 11:33:03 +10:00
Azareal
ebbfa7d247 Bumped up the version of MariaDB on Travis.
Replaced a couple of expects with expectNilErrs so we don't lose the original errors messages in those tests.
Fixed a couple more MySQL default value issues.
2018-10-07 00:02:52 +10:00
Azareal
644bba36e3 Localised more alert strings.
Renamed most of the alert phrases.
Fixes #3 - Default value issue with MySQL.
Fixed a bug where pages would't load with HTTPS disabled.
2018-10-06 23:14:11 +10:00
Azareal
6634f3f64b All the control panel routes have been moved to /routes/panel/ now.
Moved the software version to common.go for better accessibility.
2018-10-02 15:03:20 +10:00
Azareal
b2e97e47c1 Moved the last control panel theme manager route into /routes/panel/
Added the UpdateDefaultTheme function.

Renamed panelRenderTemplate to renderTemplate.
Renamed panelSuccessRedirect to successRedirect.
Reduced the amount of boilerplate in panel.GroupsEdit with renderTemplate.
Renamed the pre_render_panel_edit_group hook to pre_render_panel_group_edit.
Reduced the amount of boilerplate in panel.GroupsEditPerms with renderTemplate.
Renamed the pre_render_panel_edit_group_perms hook to pre_render_panel_group_edit_perms.

Fixed a bug where the active status for themes didn't get loaded at start-up.
Fixed a theoretical deadlock in NewThemeList.
Updated CONTRIBUTING.md
2018-10-02 14:09:17 +10:00
Azareal
322a46bcb5 Moved the control panel group routes into /routes/panel/
Added the Update, UpdatePerms and CopyPtr methods to *Group.
2018-09-30 10:08:27 +10:00
Azareal
91c81b96e0 This should fix it x.x 2018-08-29 13:46:27 +10:00
Azareal
f84f11a1eb More troubleshooting... 2018-08-29 13:36:17 +10:00
Azareal
28ba465c8a I'll try anything now o.o 2018-08-29 13:31:52 +10:00
Azareal
6839dd5ffb Experimenting with greater use of pkg/errors so we don't waste as much time as we're wasting here on pinning things down. 2018-08-29 13:23:02 +10:00
Azareal
b1c74e9069 Debugging Travis. 2018-08-29 12:22:09 +10:00
Azareal
11303d85c3 It would be really nice if the adapter didn't panic on recoverable errors x.x 2018-08-29 12:09:49 +10:00