Commit Graph

32 Commits

Author SHA1 Message Date
Azareal b04d77d7b6 optimise topic lists by caching common qcounts for getList stmts
recalc forum topic counts
reduce thaw period to 3
cache top 8 forums instead of 5
optimise GetListByForum with zero topics
add Each method to ForumStore
2020-03-09 13:51:44 +10:00
Azareal 5007ddcb1e fix tests 2020-03-08 18:10:56 +10:00
Azareal f5c9a34fdd simple top forum topiclist caching
reduce thaw period to 4
fix the _ ua bug
try using a hard period for routeAPI to avoid unneccessary churn for a return string that is unlikely to change
2020-03-08 18:04:37 +10:00
Azareal 2330b5bfeb optimise topic list orderby
filter out forums with no topics from search queries
update go version in go.mod
2020-03-03 14:25:18 +10:00
Azareal 7c3c8dcae0 unify single forums in topic lists and forum pages with GetListByForum
add GetListByForum method
remove forums with topic counts of zero from filter lists
avoid locking whenever possible in route view counter
2020-03-01 16:22:43 +10:00
Azareal b1c9e0d11c swap TopicCountInForums's implementation out for a saner one...
clean up repetition in route legends
2020-02-29 17:34:38 +10:00
Azareal ae8c27fc2d remove redundant topiccount query in topiclist slow path 2020-02-29 15:38:30 +10:00
Azareal 06f0779525 Shorten some names and eliminate a few allocs. 2019-11-12 16:56:51 +10:00
Azareal 4d9dc76392 Begin work on making the analytics panes somewhat usable when having JavaScript disabled.
Very minor refactoring here and there across the software.
Save some allocations here and there.
2019-10-28 09:13:24 +10:00
Azareal a287ccc3d6 WIP Conversations templates.
WIP Conversations structs.
WIP Unbound conversations routes.
Shorten more variable names.
Eliminate a few characters here and there.
Track down widget tick errors more easily.

Add the convos_head title phrase.
Add the convo_head title phrase.
Add the unit phrase.
Add the account_menu_messages phrase.
Add the convos_head phrase.
Add the convo_head phrase.
Add the create_convo_head phrase.
Add the create_convo_button phrase.
Add the profile_send_message phrase.
Add the panel_dashboard_disk_unknown phrase.
2019-07-27 09:29:42 +10:00
Azareal fdd223d9cf Rewrite the pagination algorithm so it works properly.
Shorten a few bits of JS.
Get the topic store methods to call BypassGet instead of duplicating logic.
2019-06-04 15:48:12 +10:00
Azareal a56ed113dd You can now filter the active memory pane by type.
Add excludes for the analytics language pane too.
Try to avoid rid queries when we already have the rids.
Use GetListByCanSee instead of GetList for preloads to increase the chances of getting something useful.
Provisionally extend topic preloading to the first three topic list pages.

Add the panel_statistics_memory_type_total phrase.
Add the panel_statistics_memory_type_stack phrase.
Add the panel_statistics_memory_type_heap phrase.
2019-05-19 11:01:11 +10:00
Azareal b9973719a5 Added a small reply cache.
Preloaded a small number of users and topics.
Use cache.Set instead of cache.Add in a few spots for topics to avoid issues with the counter falling out of sync with the cache length.
Embed Reply in ReplyUser instead of duplicating the same fields.
Add the missing AttachCount and ActionType fields to Reply.
Add the missing Poll field to TopicsRow.
Added the TopicUser.Replies method to better abstract the reply list generation logic.
Shortened some common.s to c.s
Moved memStuff out of the analytics memory panes and into analytics.js
Removed the temporary preStats fix for label overflow now that we have a better solution.
Added the panel_analytics_script_memory template to help de-dupe logic in the analytics memory panes.
Added the Topic method to TopicsRow.
Added the GetRidsForTopic method to help cache replies in a small set of scenarios.

Added the ReplyCache config.json setting.
Added the ReplyCacheCapacity config.json setting.

Added a parser test case.
Added more Reply and ReplyStore related test cases.
2019-05-17 18:40:41 +10:00
Azareal b3c4cc4bfc Fixed a bug where it would load too many forums for non-superadmins when the topic list is filtered. 2019-04-20 11:53:23 +10:00
Azareal 29d14a4a4a Added ten new plugin hooks and split topic_unknown out of the user_unknown phrase.
Added the user parameter to the action hooks to give plugins more contextual information.

Added the action_end_edit_topic plugin hook.
Added the action_end_delete_topic plugin hook.
Added the action_end_lock_topic plugin hook.
Added the action_end_unlock_topic plugin hook.
Added the action_end_stick_topic plugin hook.
Added the action_end_unstick_topic plugin hook.
Added the action_end_move_topic plugin hook.
Added the action_end_like_topic plugin hook.

Added the action_end_like_reply plugin hook.

Added the tasks_tick_topic_list plugin hook.
2019-04-19 11:02:33 +10:00
Azareal 2296008655 Basic search now works for the Search & Filter Widget. ElasticSearch has been temporarily delayed, so I can push through this update.
Added the three month time range to the analytics panes.
Began work on adding new graphs to the analytics panes.
Began work on the ElasticSearch adapter for the search system.
Added the currently limited AddKey method to the database adapters.
Expanded upon the column parsing logic in the database adapters to ease the use of InsertSelects.

Added the BulkGet method to TopicCache.
Added the BulkGetMap method to TopicStore.
TopicStore methods should now properly retrieve lastReplyBy.
Added the panel_analytics_script template to de-dupe part of the analytics logic. We plan to tidy this up further, but for now, it'll suffice.
Added plugin_sendmail and plugin_hyperdrive to the continuous integration test list.
Tweaked the width and heights of the textareas for the Widget Editor.
Added the AddKey method to *qgen.builder
Fixed a bug where using the inline forum editor would crash Gosora and wouldn't set the preset permissions for that forum properly.
Added DotBot to the user agent analytics.
Invisibles should be better handled when they're encountered now in user agent strings.
Unknown language ISO Codes in headers now have the requests fully logged for debugging purposes.
Shortened some of the pointer receiver names.
Shortened some variable names.

Added the dotbot phrase.
Added the panel_statistics_time_range_three_months phrase.

Added gopkg.in/olivere/elastic.v6 as a dependency.

You will need to run the patcher or updater for this commit.
2019-02-23 16:29:19 +10:00
Azareal d9acf27c5b The Search and Filter Widget is now partly implemented. Just Search to go in the basic implementation.
Added AJAX Pagination for the Topic List and Forum Page.
A new log file pair is now created every-time Gosora starts up.
Added proper per-theme template overrides.

Added EasyJSON to make JSON serialisation faster.
Moved a bit of boilerplate into paginator.html
Improved paginator.html with a richer template with first, last and symbols instead of text.
Phased out direct access to Templates.ExecuteTemplate across the software.
Fixed the Live Topic List so it should work again.
Added MicroAvatar to WsJSONUser for topic list JSON requests.
An instance of the plugin is now passed to plugin handlers rather than having the plugins manipulate the globals directly.
Added the pre_render_panel_forum_edit and pre_render_panel_forum_edit_perms hooks to replace pre_render_panel_edit_forum.
Renamed the pre_render_panel_edit_user hook to pre_render_panel_user_edit
Reduced the amount of noise from fsnotify.
Added RawPrepare() to qgen.Accumulator.
Added a temporary phrase whitelist to the phrase endpoint.
Moved the location of the zone data assignments in the topic list to reduce the chances of security issues in the future.
Changed the signature of routes/panel/renderTemplate() requiring some changes across the panel routes.
Removed bits of boilerplate in some of the panel routes with renderTemplate()
Added a BenchmarkTopicsGuestJSRouteParallelWithRouter benchmark.
Removed a fair bit of boilerplate for each page struct by generating a couple of interface casts for each template file instead.
Added the profile_comments_row_alt template.
Added the topics_quick_topic template to reuse part of the quick topic logic for both the topic list and forum page.
Tweaked the CSS for the Online Users Widget.
Tweaked the CSS for Widgets in every theme with a sidebar.
Refactored the template initialisers to hopefully reduce the amount of boilerplate and make things easier to maintain and follow.
Add genIntTmpl in the template initialiser file to reduce the amount of boilerplate needed for the fallback template bindings.

Removed the topics_head phrase.
Moved the paginator_ phrases into the paginator. namespace and renamed them accordingly.
Added the paginator.first_page phrase.
Added the paginator.first_page_aria phrase.
Added the paginator.last_page phrase.
Added the paginator.last_page_aria phrase.
Added the panel_forum_delete_are_you_sure phrase.

Fixed a data race in LogWarning()
2019-02-10 15:52:26 +10:00
Azareal 3465e4c08f You can now manage the attachments for an opening post by hitting edit.
The update system now uses the database as the source of truth for the last version rather than lastSchema.json
Refactored several structs and bits of code, so we can avoid allocations for contexts where we never use a relative time.
Clicking on the relative times on the topic list and the forum page should now take you to the post on the last page rather than just the last page.
Added the reltime template function.
Fixed some obsolete bits of code.
Fixed some spelling mistakes.
Fixed a bug where MaxBytesReader was capped at the maxFileSize rather than r.ContentLength.
All of the client side templates should work again now.
Shortened some statement names to save some horizontal space.
accUpdateBuilder and SimpleUpdate now use updatePrebuilder behind the scenes to simplify things.
Renamed selectItem to builder in AccSelectBuilder.
Added a Total() method to accCountBuilder to reduce the amount of boilerplate used for row count queries.
The "_builder" strings have been replaced with empty strings to help save memory, to make things slightly faster and to open the door to removing the query name in many contexts down the line.
Added the open_edit and close_edit client hooks.
Removed many query name checks.
Split the attachment logic into separate functions and de-duplicated it between replies and topics.
Improved the UI for editing topics in Nox.
Used type aliases to reduce the amount of boilerplate in tables.go and patches.go
Reduced the amount of boilerplate in the action post logic.
Eliminated a map and a slice in the topic page for users who haven't given any likes. E.g. Guests.
Fixed some long out-dated parts of the update instructions.
Updated the update instructions to remove mention of the obsolete lastSchema.json
Fixed a bug in init.js where /api/me was being loaded for guests.
Added the MiniTopicGet, GlobalCount and CountInTopic methods to AttachmentStore.
Added the MiniAttachment struct.
Split the mod floaters out into their own template to reduce duplication.
Removed a couple of redundant ParseForms.

Added the common.skipUntilIfExistsOrLine function.
Added the NotFoundJS and NotFoundJSQ functions.
Added the lastReplyID and attachCount columns to the topics table.
2018-12-27 15:42:41 +10:00
Azareal 543ad8a018 Added the login log.
Added a better paginator for view topic.
The last post on the topic list and forum pages now link to the last page of a topic for quicker navigation.
The Account Manager now utilises dyntmpl in more areas.
More tooltips.
Tweaked the colour for the validated emails in the Email Manager so it looks nicer on dark themes.
Moved some inline styles from the Email Manager into the stylesheets and removed obsolete ones.
Added the mixed BenchmarkTopicGuestAdminRouteParallelWithRouter benchmark.
The bad route benchmark should no longer abort when erroring, something which doesn't make sense as it's always expected to error.
Reduce a bit more boilerplate with renderTemplate()
Added the *CTemplateSet.addText method and used it to optimise the generated templates a tiny bit more.
The forums route now has guest and member variants generated for it.
Turned the experimental template optimisation back on for more data.

Added the routes.AccountLogins route.

Added the account_logins phrase.
Added the account_menu_logins phrase.
Added the account_logins_head phrase.
Added the account_logins_success phrase.
Added the account_logins_failure phrase.

You need to run the patcher / updater for this commit.
2018-12-17 14:58:55 +10:00
Azareal 1aac6f1268 Greatly reduced the number of w.Write calls in the generated templates.
The topic list should only be rebuilt for a short window after something related to it changes on single server setups.
Swapped out the RebuildGroupPermissions call in UpdatePerms with a more general and versable Groups.Reload call.
The template generator now use log instead of fmt for writing debug logs.
2018-11-20 09:06:15 +10:00
Azareal 17f85ceccf The time it took for the route to be processed is shown in the footer in the Nox Theme for admins now
Don't waste queries on groups without any users for the topic list.
Bypass the GzipResponseWriter in RunThemeTemplate, so we get indirected through fewer layers of Write calls.
Added an experimental flag for disabling fsnotify.
Refactored the topic benchmarks to make them a little more flexible.
Refactored a couple of benchmarks to reduce the amount of boilerplate.
Fixed a bug in the forum list for Nox where topic titles broke onto multiple lines.
Refactored the template transpiler to reduce the amount of boilerplate for generating template functions.
2018-11-17 12:36:02 +10:00
Azareal bf851bd9fc We now use Go 1.11 modules. This should help with build times, deployment and development, although it does mean that the minimum requirement for Gosora has been bumped up from Go 1.10 to Go 1.11
Added support for dyntmpl to the template system.
The Account Dashboard now sort of uses dyntmpl, more work needed here.
Renamed the pre_render_view_topic hook to pre_render_topic.
Added the GetCurrentLangPack() function.
Added the alerts_no_new_alerts phrase.
Added the account_level_list phrase.

Refactored the route rename logic in the patcher to cut down on the amount of boilerplate.
Added more route renames to the patcher. You will need to run the patcher / updater in this commit.
2018-10-27 13:40:36 +10:00
Azareal 876ae4e245 Refactored the plugin system to use a hook table. This should help to reduce the probability of data races and strange bugs, as-well as helping to abstract certain details, so future changes are less likely to break things.
Fixed the missing background for the create topic non-JS page in Nox.
The non-JS create topic page now has a localised title.
2018-10-21 23:54:32 +10:00
Azareal 859ed2c730 Finished the Most Viewed Page for Nox.
Added three phrases for topic_list.
Moved the topic_middle_inside suffix phrases from the CSS files into the template proper.
Removed a newline in topic_middle_inside and used flex instead.
The link_select drop-down is now only visible when someone clicks on the link.
Tweaked the .topic_forum line height for Cosora.
Tweaked the .topic_view_count font size for Cosora.
2018-09-26 17:46:30 +10:00
Azareal cd89e836a1 Some resource management work and performance improvements.
Added the DeallocOverflow and BulkRemove methods to UserCache.
Added the HasUser method to WsHubImpl.
Added a hot path for the common case of a single uncached item in BulkGetMap in DefaultUserStore.
2018-09-13 17:41:01 +10:00
Azareal 39adc2e606 Per-topic views are now visible in the topic pages for the Nox Theme.
The parent forum of a topic is now visible on the topic pages for the Cosora and Nox Themes.
Added basic styling for polls to the Nox Theme.
Improved the styling for topic title editing in the Nox Theme.
2018-08-30 20:31:21 +10:00
Azareal f9dedc9691 Fixed a bug with users slipping past the topic list cache. 2018-08-27 19:06:00 +10:00
Azareal 7be011a30d Almost finished live topic lists, you can find them at /topics/. You can disable them via config.json
The topic list cache can handle more groups now, but don't go too crazy with groups (e.g. thousands of them).

Make the suspicious request logs more descriptive.
Added the phrases API endpoint.
Split the template phrases up by prefix, more work on this coming up.
Removed #dash_saved and part of #dash_username.
Removed some temporary artifacts from trying to implement FA5 in Nox.
Removed some commented CSS.
Fixed template artifact deletion on Windows.
Tweaked HTTPSRedirect to make it more compact.
Fixed NullUserCache not complying with the expectations for BulkGet.
Swapped out a few RunVhook calls for more appropriate RunVhookNoreturn calls.
Removed a few redundant IsAdmin checks when IsMod would suffice.
Commented out a few pushers.
Desktop notification permission requests are no longer served to guests.
Split topics.html into topics.html and topics_topic.html
RunThemeTemplate should now fallback to interpreted templates properly when the transpiled variants aren't avaialb.e
Changed TopicsRow.CreatedAt from a string to a time.Time
Added SkipTmplPtrMap to CTemplateConfig.
Added SetBuildTags to CTemplateSet.
A bit more data is dumped when something goes wrong while transpiling templates now.
topics_topic, topic_posts, and topic_alt_posts are now transpiled for the client, although not all of them are ready to be served to the client yet.
Client rendered templates now support phrases.
Client rendered templates now support loops.
Fixed loadAlerts in global.js
Refactored some of the template initialisation code to make it less repetitive.
Split topic.html into topic.html and topic_posts.html
Split topic_alt.html into topic_alt.html and topic_alt_posts.html
Added comments for PollCache.
Fixed a data race in the MemoryPollCache.
The writer is now closed properly in WsHubImpl.broadcastMessage.
Fixed a potential deadlock in WsHubImpl.broadcastMessage.
Removed some old commented code in websockets.go

Added the DisableLiveTopicList config setting.
2018-06-24 23:49:29 +10:00
Azareal 10f4c59cb5 Fixed the Go Version in the Travis file.
Added the Go and Database versions to the Control Panel Debug Page.
Renamed common.TopicsPage to common.TopicListPage.
Renamed *HeaderVars to *Header.
Added the Paginator struct and refactored the code to use it.
io.Writers are now used instead of http.ResponseWriters in transpiled templates for greater flexibility.
Added the alert, menu_alerts, and menu_item templates.
Added support for more integer types in the arithmetic functions for the transpiled templates.
Exported AccSelectBuilder.
Added an Each method to AccSelectBuilder.
Added column quoting to the order by portions of queries for the MySQL Adapter.

Began work on the client side rendering of alerts.
Began work on the Menu Manager and associated functionality.
2018-04-22 13:33:56 +01:00
Azareal be47066770 General:
Added better pagination to /forum/{id}
Fixed the ordering of the elements in the pagesets.
Added hooks to the task system.
Reduced the amount of boilerplate for RunVhook and RunVhookNoReturn.
Added RunVhookNeedHook to allow plugin developers to add custom content types to the report system.
Added BulkGetMap to the DefaultPollStore.
Renamed the MultiServer config setting to ServerCount and made it an integer.
Don't cache topic lists for empty groups.
Don't load replies for topics without any.

Tempra Simple:
Added chart and submenu CSS.
Improved the input and select CSS.
Fixed the pageset CSS.
Fixed the poll input CSS.

Tempra Conflux:
Fixed the quick reply button CSS.
Hid the avatar in the quick reply area.
Added the poll CSS.
Fixed the pageset CSS.

Shadow:
Fixed the time range selector CSS.
Fixed the poll input CSS.
2018-02-15 13:15:27 +00:00
Azareal fa065bc584 /topics/ (the default index) is now over 25 times faster than before.
Change the length of the ticks for referrer tracking.
Fixed a bug in Shadow and Tempra Simple where the polls would appear when they shouldn't.
Added styling to Shadow for Control Panel Sub-menus.
Fixed the ridiculously wide attachment images on Cosora.
The analytics pages should no longer be treated as the dashboard style-wise.
The installer now works again.
Tests now run again, as do the benchmarks.
Refactored the topic list logic, and moved the route portion into the routes package.
Fixed an inverted comparison in the referrer logic.
Added friendly text for Mac operating systems to the english language pack.
Removed two obsolete prepared statements.
Fixed three race conditions in the database adapters.
Potentially sped up the topic page by 20%
Added several new benchmarks and refactored the others.
Removed a redundant prepared statement in user.go

Added reviseID as a primary key to the revisions table.
The user_count column was added to the users_groups table, but it's unknown if it will stay.
2018-02-10 15:07:21 +00:00
Azareal 21999cd7c6 Fixed the forum editor on Cosora. More excavating coming up.
Tweaked the preparser to hopefully fix a few problems with the WYSIWYG Editor.
Added BulkGetCopy to the ForumStores.
Added InternalErrorXML and SilentInternalErrorXML.
Tweaked the element queries so that the topic titles aren't truncated too short.

Began refactoring the topic list logic, more to come here.
2017-12-26 07:17:26 +00:00