Commit Graph

74 Commits

Author SHA1 Message Date
Azareal
67772f2f9a avoid double indirect for header in simple_forum_check_pre_perms hook
avoid double indirect for header in forum_check_pre_perms hook
use hookgen for forum_check_pre_perms hook
2020-05-26 11:46:02 +10:00
Azareal
d3b2721746 optimise attach exts
optimise PrepResources
2020-03-28 09:10:54 +10:00
Azareal
a668cd296b save bytes in the client templates
rename template file and functions to reduce bandwidth usage
replace x-loggedin with theoretically faster x-mem
remove redundant check in ua loop
move extraData initialisation down to where it's needed
2020-03-23 09:14:08 +10:00
Azareal
ade8f82af7 add python and go user agents.
add new bot class for python, go and curl to reduce resource usage.
don't push canonicals for loggedin users
try to avoid pushing google site verify when it isn't necessary
give ahrefs the semrush treatment
2020-03-22 14:37:45 +10:00
Azareal
b66a494f1c Revert "Revert "will a pointer make this faster...?""
This reverts commit 501bcc2425.
2020-03-18 19:21:34 +10:00
Azareal
501bcc2425 Revert "will a pointer make this faster...?"
This reverts commit 8e414486b8.
2020-03-15 21:34:57 +10:00
Azareal
8e414486b8 will a pointer make this faster...? 2020-03-15 17:06:44 +10:00
Azareal
46732286e5 optimise route perf tracking
expand route perf tracking to more routes
2020-02-29 11:10:39 +10:00
Azareal
ca8411a519 experimental route perf pane
add 1.14 to travis test list
more qutils tests

add panel_menu_stats_routes_perf phrase
2020-02-28 14:52:45 +10:00
Azareal
459d745cb1 initial perf anaytics
add tasks to debug page
ignore .git on debug page for speed

add perfchunks table

Renamed phrases (changed statistics to stats):
panel_menu_stats
panel_menu_stats_posts
panel_menu_stats_topics
panel_menu_stats_forums
panel_menu_stats_routes
panel_menu_stats_agents
panel_menu_stats_systems
panel_menu_stats_languages
panel_menu_stats_referrers
panel_menu_stats_memory
panel_menu_stats_active_memory
panel_menu_stats_perf
panel_stats_views_head_suffix
panel_stats_user_agents_head
panel_stats_forums_head
panel_stats_languages_head
panel_stats_post_counts_head
panel_stats_referrers_head
panel_stats_routes_head
panel_stats_operating_systems_head
panel_stats_topic_counts_head
panel_stats_requests_head
panel_stats_memory_head
panel_stats_active_memory_head
panel_stats_spam_hide
panel_stats_spam_show
panel_stats_memory_type_total
panel_stats_memory_type_stack
panel_stats_memory_type_heap
panel_stats_time_range_one_year
panel_stats_time_range_three_months
panel_stats_time_range_one_month
panel_stats_time_range_one_week
panel_stats_time_range_two_days
panel_stats_time_range_one_day
panel_stats_time_range_twelve_hours
panel_stats_time_range_six_hours
panel_stats_post_counts_chart_aria
panel_stats_topic_counts_chart_aria
panel_stats_requests_chart_aria
panel_stats_memory_chart_aria
panel_stats_details_head
panel_stats_post_counts_table_aria
panel_stats_topic_counts_table_aria
panel_stats_route_views_table_aria
panel_stats_requests_table_aria
panel_stats_memory_table_aria
panel_stats_views_suffix
panel_stats_posts_suffix
panel_stats_topics_suffix
panel_stats_user_agents_no_user_agents
panel_stats_forums_no_forums
panel_stats_languages_no_languages
panel_stats_post_counts_no_post_counts
panel_stats_referrers_no_referrers
panel_stats_routes_no_routes
panel_stats_operating_systems_no_operating_systems
panel_stats_memory_no_memory

Added phrases:
panel_debug_tasks
panel_debug_tasks_half_second
panel_debug_tasks_second
panel_debug_tasks_fifteen_minute
panel_debug_tasks_hour
panel_debug_tasks_shutdown
panel_stats_perf_head
panel_stats_perf_low
panel_stats_perf_high
panel_stats_perf_avg

You will need to run the updater / patcher for this commit.
2020-02-23 19:08:47 +10:00
Azareal
0d81557009 Add DisableLastIP configuration setting.
Reduce default PollIPCutoff to 365.
2020-01-02 16:04:10 +10:00
Azareal
35ddc89009 Optimise the database layer.
Refactor database adapters.
Experimental last ip cutoff.

More parser test cases.
2020-01-01 07:57:54 +10:00
Azareal
501c0dff0a Shorten things. 2019-12-07 16:27:01 +10:00
Azareal
142359ce11 Add SslSchema config setting.
Reject URL usernames on registration.

Reduce length of char variable name.
2019-11-04 21:55:52 +10:00
Azareal
65b4fef40e WIP Add more language strings for panel debug.
Shorten more things.
Sneak draft tables in.

Add the profile_block phrase.
Add the panel_debug_memory_stats phrase.
Add the panel_debug_memory_stats_sys phrase.
Add the panel_debug_memory_stats_heapsys phrase.
Add the panel_debug_memory_stats_heapalloc phrase.
Add the panel_debug_memory_stats_heapidle phrase.
Add the panel_debug_memory_stats_heapobjects phrase.
Add the panel_debug_memory_stats_stackinuse phrase.
Add the panel_debug_caches phrase.
Add the panel_debug_caches_topic phrase.
Add the panel_debug_caches_user phrase.
Add the panel_debug_caches_reply phrase.
Add the panel_debug_caches_topic_list phrase.
Add the panel_debug_database phrase.
Add the panel_debug_database_topics phrase.
Add the panel_debug_database_users phrase.
Add the panel_debug_database_replies phrase.
Add the panel_debug_disk phrase.
2019-10-11 10:57:33 +10:00
Azareal
218e177a29 Shorten more things. 2019-09-01 08:59:00 +10:00
Azareal
c309faf79f Use constant time comparisons for sessions in forms. 2019-08-30 20:59:50 +10:00
Azareal
1c0a3dd5b6 Accept shorter session param names as these are so common.
Prealloc in DefaultPollStore.BulkGetMap
Shorten more variable names.
Remove a bit of inline CSS in topic_alt_quick_reply
2019-07-27 08:36:06 +10:00
Azareal
89c8f4e775 Use a client template for poll input.
Tweak the poll input phrase.
Increase the padding on .pollinput for Nox.
2019-06-16 17:26:31 +10:00
Azareal
ee3c29b136 Let admins change and revoke user avatars.
Linkify avatars in the user manager too.

Add the panel_user_avatar phrase.
2019-06-11 08:00:57 +10:00
Azareal
05c2ac3ce4 Add the disk and database sections to the debug page.
Add the ActivityStream interface to abstract Get, Add and Count.
Rename the GlobalCount methods to Count for simplicity.
Simplify the variable names in the Count methods.
Rename the GlobalCount method to Count and rename the original Count method to CountUser in LoginLogStore.
Add a float64 case for bunit, sort of.
Theme.RunTmpl now returns ErrBadDefaultTemplate instead of panicking when an interpreted template doesn't exist.
Widget.Allowed now checks the zoneid.
Fire the alert off in the background in AddActivityAndNotifyTarget instead of blocking the request.
Use ErrBadDefaultTemplate instead of calling DefaultTemplates.Lookup directly for custom pages.
Split the page struct for the debug page into multiple structs to make things more organised.

Add the Count method to ProfileReplyStore.
Add the Count method to ReplyStore.
Add the DirSize utility function.

Add a few ActivityStream tests.

Secret gallery stuff.
2019-06-01 22:31:48 +10:00
Azareal
c8948a778c Oops, need to generate a cached copy of each theme's lists in Hyperdrive.
Renamed common.getTheme to common.GetThemeByReq.
2019-04-29 22:23:17 +10:00
Azareal
bd7b1681e3 Hyperdrive now caches /forums/ too.
Added the route_forum_list_start plugin hook.
2019-04-28 20:50:45 +10:00
Azareal
27a4a74840 You can now re-order forums by dragging them in the Forum Manager.
Added some visual and textual hints to make it clearer that Menu Items and Forums can be dragged.
Added a hint to flush the page after pushing the <head>
Added the notice client template and pushNotice client function.

Used a pointer instead of a struct for AnalyticsTimeRange in the analytics routes.
Caught a potential missing error check in InitPhrases.
Use struct{} instead of bool in some of the user mapping maps for WebSockets to save space.
Added the buildUserExprs function to eliminate a bit of duplication.
Fixed a typo in ForumsEdit where it referenced a non-existent notice phrase.
Client hooks can now sort of return things.
Panel phrases are now fetched by init.js, but only in the control panel.
Reduced the number of unused phrases loaded in both the front-end and the control panel.

Plugin hyperdrive should handle Gzip better now.

Added the panel.ForumsOrderSubmit route.

Added the panel_hints_reorder phrase.
Moved the panel_forums phrases into the panel. namespace.
Added the panel.forums_order_updated phrase.
Renamed the panel_themes_menus_item_edit_button_aria phrase to panel_themes_menus_items_edit_button_aria
Renamed the panel_themes_menus_item_delete_button_aria phrase to panel_themes_menus_items_delete_button_aria
Added the panel_themes_menus_items_update_button phrase.

You will need to run the patcher / updater for this commit.
2019-04-27 16:32:26 +10:00
Azareal
42b9f27c45 Defer setting headers to renderTemplate to avoid wasting resources for JSON requests and for future optimisations.
Error pages now use routes.renderTemplate bringing them in line with the other standard routes.
Every route should use a renderTemplate function now instead of calling RunTmpl.
Merged some pi and renderTemplate lines in routes to reduce the amount of vertical space used.
2019-04-19 19:34:16 +10:00
Azareal
9179eb9711 The moderation logs can now be localised.
Error pages which call CustomError should now stick on the theme the user chose rather than the global fallback theme.
Move the theme fetching logic out of userCheck and panelUserCheck and into it's own function.
Split prepResources out of userCheck, so that errors can make use of it.

Fixed a bug where no scripts or stylesheets would run on error pages.
Fixed a bug where unknown topics would point to /user/ rather than /topic/

Added the panel_logs_moderation_action_topic_stick phrase.
Added the panel_logs_moderation_action_topic_unstick phrase.
Added the panel_logs_moderation_action_topic_lock phrase.
Added the panel_logs_moderation_action_topic_unlock phrase.
Added the panel_logs_moderation_action_topic_delete phrase.
Added the panel_logs_moderation_action_topic_move phrase.
Added the panel_logs_moderation_action_topic_move_dest phrase.
Added the panel_logs_moderation_action_topic_unknown phrase.
Added the panel_logs_moderation_action_reply_delete phrase.
Added the panel_logs_moderation_action_user_ban phrase.
Added the panel_logs_moderation_action_user_unban phrase.
Added the panel_logs_moderation_action_user_activate phrase.
Added the panel_logs_moderation_action_unknown phrase.
Added the user_unknown phrase.
2019-04-18 11:00:39 +10:00
Azareal
546aa5f845 The attachment managers for replies are now rendered with client templates
Avoid loading the topic phrases when the user isn't logged in.
Don't load the topic_c_edit_post and topic_c_attach_item client templates when the user isn't logged in.
Fixed a bug where multiple attachment managers wouldn't load.

Added the topic_c_attach_item template.
2019-04-15 11:54:13 +10:00
Azareal
cdb2f0711d Attachments continue to function after the parent topic is moved now.
Changed the signature of AttachmentStore.Add
Added the MoveTo method to AttachmentStore.
Added the MoveToByExtra method to AttachmentStore.
Don't forget to actually add the pre script for topic_c_edit_post x.x

Added the extra column to the attachments table.

You will need to run the updater / patcher for this commit.
2019-04-13 21:54:22 +10:00
Azareal
733a802357 Use the langpack's isocode in header.html instead of en.
Added the GetLangPack function.
2019-04-09 17:21:00 +10:00
Azareal
e1dfebf090 main.css should be invalidated properly too. 2019-03-25 14:01:50 +10:00
Azareal
660f24acff Experimenting with Nox's Control Panel.
Experimenting with better cache busting for static resources.
HTTPSRedirect requests are now counted in the route analytics.
More scripts are loaded asynchronously now.
Upped the default ReadTimeout to eight seconds.
Reduce the number of unneccesary NewAcc calls.
Added panel_before_head as an injection point for themes.
Themes can now declare scripts to be loaded asynchronously.
Tweaked the WS resumption algorithm to mae the backoffs a little less aggressive.
Fixed an ordering issue in the WS resumption algorithm where backoffs weren't expiring as fast as they should have.
Fixed a bug where template logs weren't being written due to a panic.
You can now use byte slices in more places in the transpiled templates.
Fixed a bug where Cosora's misc.js seemed to be erroring out.
Fixed a bug where YT embeds were getting blocked by the CSP.

Added the panel_back_to_site phrase.
Added the panel_welcome phrase.
2019-03-22 08:59:41 +10:00
Azareal
93b292acc0 Added a little note to the installation guide about setting up emails.
Alerts will be resynced whenever a WebSockets connection is re-established now.
Fixed client side overrides.
2019-03-05 15:53:02 +10:00
Azareal
a4b579d75d Whoops, this should fix the SIGSEGV. 2019-02-28 17:34:56 +10:00
Azareal
cc1d0f089a Client templates can now be overriden like every other template.
The client templates load earlier now for a smoother user experience.
Added a setting for setting a Google Site Verification meta tag without editing header.html
Added support for favicons. /static/favicon.ico will be mapped to favicon.ico, if it exists.
The parent forum is now visible on the topics list for Nox.

Language headers which contain the wildcard character are no longer considered unknowns.
Meta descriptions and open graph descriptions are no longer emitted for logged in users.
Slimmed down topics_topic slightly for Nox.
Pre-parsed widgets are now minified.
Stale WebSockets connections should be cleaned up far quicker now.
Template generation is now logged separately.
Commented out some obsolete template logic.
Marked a few template generator fields as unexported.

Fixed the styling for the ban page in the profile for Nox.
Fixed the styling for colline for Cosora and Tempra Simple.
Fixed the sidebar overflowing outside of the box on Nox.
Fixed the meta description text overflowing the box in the Setting Manager on Nox.
Fixed excessive padding in the Page Manager.
Fixed a few missing border on the profiles for Tempra Simple.
Fixed the sidebar appearing in places it shouldn't on Tempra Simple.
Fixed the status code emitted by NotFoundJS
Fixed a bug where Gosora kept falling back to interpreted templates.
Fixed a bug where WebSockets connections weren't getting closed properly if the user cache overflowed.
Fixed a bug where WebSocket connections weren't getting initialised for guests.
Fixed a bug where template overrides weren't always getting applied.
Fixed a bug where root template overrides weren't always getting applied.

Added the google_site_verify setting.
Added the google_site_verify phrase.

You will need to run the patcher or updater for this commit.
2019-02-28 17:28:17 +10:00
Azareal
3465e4c08f You can now manage the attachments for an opening post by hitting edit.
The update system now uses the database as the source of truth for the last version rather than lastSchema.json
Refactored several structs and bits of code, so we can avoid allocations for contexts where we never use a relative time.
Clicking on the relative times on the topic list and the forum page should now take you to the post on the last page rather than just the last page.
Added the reltime template function.
Fixed some obsolete bits of code.
Fixed some spelling mistakes.
Fixed a bug where MaxBytesReader was capped at the maxFileSize rather than r.ContentLength.
All of the client side templates should work again now.
Shortened some statement names to save some horizontal space.
accUpdateBuilder and SimpleUpdate now use updatePrebuilder behind the scenes to simplify things.
Renamed selectItem to builder in AccSelectBuilder.
Added a Total() method to accCountBuilder to reduce the amount of boilerplate used for row count queries.
The "_builder" strings have been replaced with empty strings to help save memory, to make things slightly faster and to open the door to removing the query name in many contexts down the line.
Added the open_edit and close_edit client hooks.
Removed many query name checks.
Split the attachment logic into separate functions and de-duplicated it between replies and topics.
Improved the UI for editing topics in Nox.
Used type aliases to reduce the amount of boilerplate in tables.go and patches.go
Reduced the amount of boilerplate in the action post logic.
Eliminated a map and a slice in the topic page for users who haven't given any likes. E.g. Guests.
Fixed some long out-dated parts of the update instructions.
Updated the update instructions to remove mention of the obsolete lastSchema.json
Fixed a bug in init.js where /api/me was being loaded for guests.
Added the MiniTopicGet, GlobalCount and CountInTopic methods to AttachmentStore.
Added the MiniAttachment struct.
Split the mod floaters out into their own template to reduce duplication.
Removed a couple of redundant ParseForms.

Added the common.skipUntilIfExistsOrLine function.
Added the NotFoundJS and NotFoundJSQ functions.
Added the lastReplyID and attachCount columns to the topics table.
2018-12-27 15:42:41 +10:00
Azareal
27fb79d4d5 Fixed a bug where forums inheriting from the group didn't appear on the forum list and the topic list.
Cleaned up the logic in preRoute slightly.
Reordered the logic in preRoute so that the CSP is set for the PreError too.
Clear a user's cached data when their IP changes.
Hide the comment header when there aren't any comments.
2018-11-29 17:27:17 +10:00
Azareal
17f85ceccf The time it took for the route to be processed is shown in the footer in the Nox Theme for admins now
Don't waste queries on groups without any users for the topic list.
Bypass the GzipResponseWriter in RunThemeTemplate, so we get indirected through fewer layers of Write calls.
Added an experimental flag for disabling fsnotify.
Refactored the topic benchmarks to make them a little more flexible.
Refactored a couple of benchmarks to reduce the amount of boilerplate.
Fixed a bug in the forum list for Nox where topic titles broke onto multiple lines.
Refactored the template transpiler to reduce the amount of boilerplate for generating template functions.
2018-11-17 12:36:02 +10:00
Azareal
9f273a99f5 Trying to reduce the amount of UserCheck() boilerplate in the routes.
Reduced the amount of boilerplate in routes with renderTemplate()
Reduced the amount of boilerplate in routes with ParseSEOURL()
Removed some dated commented bits of code.
Used StashConfig in a few more places in the benchmarks to reduce the amount of boilerplate.

Renamed the pre_render_forum_list hook to pre_render_forums.
Renamed the pre_render_topic_list hook to pre_render_topics.
Renamed a few benchmark variables to simplify the code.
2018-11-12 19:23:36 +10:00
Azareal
35b1613005 Make the level bars in the profiles look less ugly. 2018-10-24 22:54:07 +10:00
Azareal
876ae4e245 Refactored the plugin system to use a hook table. This should help to reduce the probability of data races and strange bugs, as-well as helping to abstract certain details, so future changes are less likely to break things.
Fixed the missing background for the create topic non-JS page in Nox.
The non-JS create topic page now has a localised title.
2018-10-21 23:54:32 +10:00
Azareal
c15f02396a Fixed a bug where a user's tag can get desynced with their group. 2018-10-14 16:16:42 +10:00
Azareal
34b8d8dc3c Fixed a bug where group permissions didn't update until Gosora was restarted.
The profile titles can now be localised.
The profile owner tag can now be localised.

Added some changes I forgot to push in the previous commit.
2018-10-14 16:09:25 +10:00
Azareal
644bba36e3 Localised more alert strings.
Renamed most of the alert phrases.
Fixes #3 - Default value issue with MySQL.
Fixed a bug where pages would't load with HTTPS disabled.
2018-10-06 23:14:11 +10:00
Azareal
41c3a5bb4a Added a Content Security Policy for upgrading insecure images to HTTPS to avoid mixed content warnings. 2018-08-30 15:57:07 +10:00
Azareal
d8682aca94 Move these header sets to the start of the router. 2018-08-15 18:06:48 +10:00
Azareal
a9a1d667c0 Added the /api/me/ API endpoint.
Moved a script block into an API endpoint.
Split the gigantic switch in ServeHTTP into it's own function.
Scripts should load more efficiently now.
2018-08-12 01:53:42 +10:00
Azareal
b8681524f3 Added a mitigation for polyglot attacks.
.panel_floater is now floated to the right on Nox.
2018-08-06 12:36:24 +10:00
Azareal
01a692ab5b Added the word filter store and moved the word filter routes into the route package.
Added tests for the word filter store.
Added qgen.NewAcc() to reduce the amount of boilerplate needed for creating an accumulator.
Exposed the RecordError method on the accumulator.
Added an Add method to PluginList and removed AddPlugin() in favour of that.

More panel buttons on Nox should be styled now.
Added the panel_update_button_text phrase for future use.

More errors might be caught in the thumbnailer now.
Removed ls from .travis.yml, it was there for debugging Code Climate.
2018-08-04 21:46:36 +10:00
Azareal
3043ce083d Basic proxy IP support.
Switched out a couple of <a>s in the Control Panel with <h1>s, we might want to use <h2> here instead.
Added graphs for Nox.
Tweaked header markup for Nox.
Added support for live topic list to Nox.
Added styling for stickies on Nox.
Fixed topic list text overflow on Nox.
2018-07-03 20:01:49 +10:00
Azareal
7be011a30d Almost finished live topic lists, you can find them at /topics/. You can disable them via config.json
The topic list cache can handle more groups now, but don't go too crazy with groups (e.g. thousands of them).

Make the suspicious request logs more descriptive.
Added the phrases API endpoint.
Split the template phrases up by prefix, more work on this coming up.
Removed #dash_saved and part of #dash_username.
Removed some temporary artifacts from trying to implement FA5 in Nox.
Removed some commented CSS.
Fixed template artifact deletion on Windows.
Tweaked HTTPSRedirect to make it more compact.
Fixed NullUserCache not complying with the expectations for BulkGet.
Swapped out a few RunVhook calls for more appropriate RunVhookNoreturn calls.
Removed a few redundant IsAdmin checks when IsMod would suffice.
Commented out a few pushers.
Desktop notification permission requests are no longer served to guests.
Split topics.html into topics.html and topics_topic.html
RunThemeTemplate should now fallback to interpreted templates properly when the transpiled variants aren't avaialb.e
Changed TopicsRow.CreatedAt from a string to a time.Time
Added SkipTmplPtrMap to CTemplateConfig.
Added SetBuildTags to CTemplateSet.
A bit more data is dumped when something goes wrong while transpiling templates now.
topics_topic, topic_posts, and topic_alt_posts are now transpiled for the client, although not all of them are ready to be served to the client yet.
Client rendered templates now support phrases.
Client rendered templates now support loops.
Fixed loadAlerts in global.js
Refactored some of the template initialisation code to make it less repetitive.
Split topic.html into topic.html and topic_posts.html
Split topic_alt.html into topic_alt.html and topic_alt_posts.html
Added comments for PollCache.
Fixed a data race in the MemoryPollCache.
The writer is now closed properly in WsHubImpl.broadcastMessage.
Fixed a potential deadlock in WsHubImpl.broadcastMessage.
Removed some old commented code in websockets.go

Added the DisableLiveTopicList config setting.
2018-06-24 23:49:29 +10:00