Commit Graph

53 Commits

Author SHA1 Message Date
Azareal e9819389af cover avatars with AvatarResBase config setting 2020-07-31 22:45:20 +10:00
Azareal 4680aef783 set RefNoRef to true when using an external domain for StaticResBase 2020-07-31 15:53:33 +10:00
Azareal 26ad61057a cover dynamically loaded static resources
cover default noavatars
2020-07-31 09:40:14 +10:00
Azareal 43d72e6f3b wip allow for more cdns
add res template function
add ExtraCSPOrigins config setting
add StaticResBase config setting
skip flush directives
2020-07-30 18:10:29 +10:00
Azareal ea1037bd63 track favicon stats
experiment with tracking average route performance
temporary error route stub
optimise dumprequest
add DisableAnalytics config setting
fix double hyphens in slugs being mistaken for sql injection
more querygen tests

You wil need to run the updater / patcher for this commit.
2020-02-26 20:34:38 +10:00
Azareal 7fd67210d0 reduce poll ip cutoff to 90 days 2020-02-19 15:00:55 +10:00
Azareal da3a41b51a reduce poll ip cutoff to 180 days 2020-02-19 14:58:45 +10:00
Azareal e37c98eaa1 cache default noavatar strings to avoid allocs 2020-02-07 22:27:33 +10:00
Azareal a569772e9c Add DisableRegLog configuration setting.
Unit tests for DisablePostIP. Awkward but does the job for now.
2020-01-04 15:30:25 +10:00
Azareal b39d90e4b6 Add DisablePostIP configuration setting. 2020-01-03 07:52:41 +10:00
Azareal aed409ae9e Add DisablePollIP configuration setting. 2020-01-02 19:49:34 +10:00
Azareal 0d81557009 Add DisableLastIP configuration setting.
Reduce default PollIPCutoff to 365.
2020-01-02 16:04:10 +10:00
Azareal e2f97d29cb Experiment with 720 days for default poll vote cutoff. 2020-01-02 15:48:05 +10:00
Azareal f789a8498e Add PollIPCutoff configuration setting.
Add tests for qgen processWhere and qgen mysql buildWhere.

Shorten some things.
2020-01-02 15:28:36 +10:00
Azareal 35ddc89009 Optimise the database layer.
Refactor database adapters.
Experimental last ip cutoff.

More parser test cases.
2020-01-01 07:57:54 +10:00
Azareal d22021b022 Reduce default PostIPCutoff from 180 to 120 days.
Reduce default LogPruneCutoff from 365 to 180 days.
2019-12-19 17:09:56 +10:00
Azareal 0dede6a329 NoEmbeds privacy and config settings.
Phase out url_prefix and url_name.
Eliminate some boilerplate and allocations while we're at it.
Reduce the number of conditional statements in url validator loops.
Better parsing of Site.URL to handle user error better.

You will have to run the patcher / updater for this commit.
2019-12-08 13:40:56 +10:00
Azareal 142359ce11 Add SslSchema config setting.
Reject URL usernames on registration.

Reduce length of char variable name.
2019-11-04 21:55:52 +10:00
Azareal 4d8c97812d More conversations work.
Add support for AS in columns for SimpleInnerJoin.
Add a referrer policy to improve privacy a little.
Shorten /static/ to /s/ since it comes up so much.
Remove some obsolete code.
Shorten some variable names.
Reduce the amount of boilerplate in the patcher.

Added the RefNoTrack and RefNoRef privacy config settings.

You may need to run the updater / patcher for this commit.
2019-08-14 20:39:04 +10:00
Azareal a5441f18de More conversations progress, still not usable yet.
Flush the debug page incrementally.
Eliminate a tiny bit of code in PreparseMessage.
Shorten some of the pointer capture variables in the poll store.
Tweak the advanced installation instructions for EasyJSON to make it more repeatable.

Added the ConvoKey config setting.
Added /.git/ disk usage to the debug page, if it exists.
Added a UserStore.BulkGetMap multi-user test.
2019-06-16 16:30:58 +10:00
Azareal ac0dd6d2cb The login and registration logs are pruned after a year now.
Add filter exclusions for the user agents analytics pane.
Shorten some of the variable names in qgen.

Add ComplexDelete to the query adapter.
Add DateCutoff and DateOlderThan to accDeleteBuilder.
2019-06-05 14:57:10 +10:00
Azareal 0d1e4e5993 Add super experimental support for per-forum templates.
Add the ExtraTmpls developer setting to config.json
Call ForumStore.BypassGet in ForumStore.Reload instead of duplicating it.
Fix a potential deadlock in ForumStore.Create.
Attempt to fix a weird installation bug.
Add a few more forum store test cases.

You will need to run the updater / patcher for this commit.
2019-05-27 19:00:40 +10:00
Azareal b9973719a5 Added a small reply cache.
Preloaded a small number of users and topics.
Use cache.Set instead of cache.Add in a few spots for topics to avoid issues with the counter falling out of sync with the cache length.
Embed Reply in ReplyUser instead of duplicating the same fields.
Add the missing AttachCount and ActionType fields to Reply.
Add the missing Poll field to TopicsRow.
Added the TopicUser.Replies method to better abstract the reply list generation logic.
Shortened some common.s to c.s
Moved memStuff out of the analytics memory panes and into analytics.js
Removed the temporary preStats fix for label overflow now that we have a better solution.
Added the panel_analytics_script_memory template to help de-dupe logic in the analytics memory panes.
Added the Topic method to TopicsRow.
Added the GetRidsForTopic method to help cache replies in a small set of scenarios.

Added the ReplyCache config.json setting.
Added the ReplyCacheCapacity config.json setting.

Added a parser test case.
Added more Reply and ReplyStore related test cases.
2019-05-17 18:40:41 +10:00
Azareal e76223224a Implemented periodic post IP purging for replies.
Update for MySQL now supports DateCutoff and DateOlderThan.
Added the DateOlderThan method to accUpdateBuilder.

Added the PostIPCutoff config setting.
2019-05-08 19:50:10 +10:00
Azareal a1161e20f3 The rest of the Control Panel (exc. the Group Editor) now uses dyntmpl.
Improved the noavatar cacheability by constraining them in a range.
Improved the noavatar cacheability by serving a small subset from Gosora.
Improved the formatting of bytes in the memory analytics pane.
Static resources with checksums will now have their caches expire in a week rather than a day.

Tweaked the padding on the sub_heads on Nox.
Moved a block of CSS out of a template and into panel.css in Tempra Simple and Shadow.

Added the panel_themes_menus_items_suffix phrase.

Added the DisableNoavatarRange config setting.
Added the DisableDefaultNoavatar config setting.
2019-05-03 18:34:18 +10:00
Azareal a7e396cbc2 Added the LoosePort config.json setting. 2019-04-14 10:29:36 +10:00
Azareal 8bfd5adbbb Added the LooseHost config setting.
Malformed requests are now logged with an integer denoting the block of code which triggered it.
2019-04-14 10:13:32 +10:00
Azareal 1115c0a4c5 Added the Plugin section to config.json
Added support for ZIP files to the backups route.
The post counter won't be incremented if json marshalling fails now.

Added the router_pre_route hook.
Added the router_end hook.
Added the action_end_create_topic hook.
Added the action_end_create_reply hook.
2019-04-06 11:08:49 +10:00
Azareal af9a56a9a9 Server push is back. But only for Chrome and Firefox.
Added the DisableServerPush and EnableCDNPush config.json settings.
2019-04-02 17:43:11 +10:00
Azareal 9d321e9f23 The router now redirects requests to localhost domains with localhost equivalents in the host header which don't quite match the destination to the proper domain.
The router now rejects host headers with the wrong port for non-standard ports.
The www. redirect now handles non-standard ports properly.
The Site.Port configuration setting is now validated on start-up to ensure it's a valid integer.

Quickly fixed up the grammar of the Port block in configuration.md
2019-04-01 15:44:38 +10:00
Azareal f2572fc3bd Fixed the analytics charts.
Fixed the back to site link destination.

Added the ReadTimeout, WriteTimeout, IdleTimeout and FullReqLog config settings.
2019-03-16 22:50:31 +10:00
Azareal 41e7464f8d Added support for SMTP TLS. 2019-03-03 14:00:04 +10:00
Azareal fe33112827 Added an experimental content security policy.
Added support for Open Graph Descriptions.
Nox now officially supports notices.
Tweaked the language detection algorithm to cover more cases.
Tweaked the user agent parser to accomodate DotBot better.
Added a non-JS fallback for the theme selector.
Tweaked the padding on widget simple.
Scripts should now execute properly for individual language charts in the analytics panel.
2019-02-24 18:02:00 +10:00
Azareal cf4a9d1738 You can now disable the search subsystem.
Try Travis with MariaDB 12...
2019-02-23 16:55:34 +10:00
Azareal 5db5bc0c7e Reply attachments can be managed too now.
Added eight database indices.
Fixed a bug where the second tick wouldn't fire.
Tweaked the .topic_forum in Nox by a pixel.
Replaced some _installer strings with blank strings for consistency with the builders.
Greatly reduced the number of allocations in the user agent parser.
Added ampersand entities in more attachment URLs to avoid accidental mangling.
.edit_source is now hidden for guests.
Guest noavatars are now pre-calculated to reduce the number of allocations.
Lazily initialised a couple of maps in ViewTopic to reduce the number of unnecessary allocations slightly.

Added the unsafe BytesToString function. Please don't use this, if you don't have to.
Added the AddIndex method to the adapter and associated components.
Added the /reply/attach/add/submit/ route.
Added the /reply/attach/remove/submit/ route.
Added the topic_alt_userinfo template.
Replaced Attachments.MiniTopicGet with MiniGetList.
Added Attachments.BulkMiniGetList.

Added a quick test for ReplyStore.Create.
Added BenchmarkPopulateTopicWithRouter.
Added BenchmarkTopicAdminFullPageRouteParallelWithRouter.
Added BenchmarkTopicGuestFullPageRouteParallelWithRouter.

You will need to run the updater or patcher for this commit.
2018-12-31 19:03:49 +10:00
Azareal 17f85ceccf The time it took for the route to be processed is shown in the footer in the Nox Theme for admins now
Don't waste queries on groups without any users for the topic list.
Bypass the GzipResponseWriter in RunThemeTemplate, so we get indirected through fewer layers of Write calls.
Added an experimental flag for disabling fsnotify.
Refactored the topic benchmarks to make them a little more flexible.
Refactored a couple of benchmarks to reduce the amount of boilerplate.
Fixed a bug in the forum list for Nox where topic titles broke onto multiple lines.
Refactored the template transpiler to reduce the amount of boilerplate for generating template functions.
2018-11-17 12:36:02 +10:00
Azareal 218070fceb Added the DisableJSAntispam config.json setting. 2018-10-16 20:02:51 +10:00
Azareal 0a628f7201 PNG and JPG avatars are now encoded as JPG images leading to a dramatic drop in the amount of bandwidth used.
Did some work on image thumbnailing, but our dependencies are acting up delaying this from being released.

Fixed the positions of the topic list bits for Nox on mobile.
Removed APNG as an accepted image format, as we don't currently have a good way of optimising these images.
Added a comment regarding the constant time compare for sessions.
Added a warning about putting Gosora in www folders.
Noavatars can now take a width parameters.
Added a bit of missing validation for the avatar uploader.
Refactored the multiple file detector for the avatar uploader.

Added a Run method to accDeleteBuilder.
Added an EachInt method to AccSelectBuilder.
Added a Run method to accInsertBuilder.

Added the users_avatar_queue table, you will need to run the patcher / update script.
You might also want to update the Noavatar field in your config.json file with the new one.
2018-07-28 22:52:23 +10:00
Azareal 7be011a30d Almost finished live topic lists, you can find them at /topics/. You can disable them via config.json
The topic list cache can handle more groups now, but don't go too crazy with groups (e.g. thousands of them).

Make the suspicious request logs more descriptive.
Added the phrases API endpoint.
Split the template phrases up by prefix, more work on this coming up.
Removed #dash_saved and part of #dash_username.
Removed some temporary artifacts from trying to implement FA5 in Nox.
Removed some commented CSS.
Fixed template artifact deletion on Windows.
Tweaked HTTPSRedirect to make it more compact.
Fixed NullUserCache not complying with the expectations for BulkGet.
Swapped out a few RunVhook calls for more appropriate RunVhookNoreturn calls.
Removed a few redundant IsAdmin checks when IsMod would suffice.
Commented out a few pushers.
Desktop notification permission requests are no longer served to guests.
Split topics.html into topics.html and topics_topic.html
RunThemeTemplate should now fallback to interpreted templates properly when the transpiled variants aren't avaialb.e
Changed TopicsRow.CreatedAt from a string to a time.Time
Added SkipTmplPtrMap to CTemplateConfig.
Added SetBuildTags to CTemplateSet.
A bit more data is dumped when something goes wrong while transpiling templates now.
topics_topic, topic_posts, and topic_alt_posts are now transpiled for the client, although not all of them are ready to be served to the client yet.
Client rendered templates now support phrases.
Client rendered templates now support loops.
Fixed loadAlerts in global.js
Refactored some of the template initialisation code to make it less repetitive.
Split topic.html into topic.html and topic_posts.html
Split topic_alt.html into topic_alt.html and topic_alt_posts.html
Added comments for PollCache.
Fixed a data race in the MemoryPollCache.
The writer is now closed properly in WsHubImpl.broadcastMessage.
Fixed a potential deadlock in WsHubImpl.broadcastMessage.
Removed some old commented code in websockets.go

Added the DisableLiveTopicList config setting.
2018-06-24 23:49:29 +10:00
Azareal f8f46b3c48 Added support for two-factor authentication.
Added the Account Dashboard and merged a few account views into it.
BREAKING CHANGE: We now use config/config.json instead of config/config.go, be sure to setup one of these files, you can config_default.json as an example of what a config.json should look like. If you don't have an existing installation, you can just rely on the installer to do this for you.

CSS Changes (does not include Nox Theme):
Sidebar should no longer show up in the account manager in some odd situations or themes.
Made a few CSS rules more generic.
Forms have a new look in Cosora now.

Config Changes:
Removed the DefaultRoute config field.
Added the DefaultPath config field.
Added the MaxRequestSizeStr config field to make it easier for users to input custom max request sizes without having to use a calculator or figure out how many bytes there are in a megabyte.
Removed the CacheTopicUser config field.
Added the UserCache config field.
Added the TopicCache config field

Phrases:
Removed ten english phrases.
Added 21 english phrases.
Changed eleven english phrases.
Removed some duplicate indices in the english phrase pack.

Removed some old benchmark code.
Tweaked some things to make the linter happy.
Added comments for all the MemoryUserCache and MemoryTopicCache methods.
Added a comment for the null caches, consult the other caches for further information on the methods.
Added a client-side check to make sure the user doesn't upload too much data in a single post. The server already did this, but it might be a while before feedback arrives from it.
Simplified a lot of the control panel route code with the buildBasePage function.
Renamed /user/edit/critical/ to /user/edit/password/
Renamed /user/edit/critical/submit/ to /user/edit/password/submit/
Made some small improvements to SEO with a couple of meta tags.
Renamed some of the control panel templates so that they use _ instead of -.
Fixed a bug where notices were being moved to the wrong place in some areas in Cosora.
Added the writeJsonError function to help abstract writing json errors.
Moved routePanelUsers to panel.Users
Moved routePanelUsersEdit to panel.UsersEdit
Moved routePanelUsersEditSubmit to panel.UsersEditSubmit
Renamed routes.AccountEditCritical to routes.AccountEditPassword
Renamed routes.AccountEditCriticalSubmit to routes.AccountEditPasswordSubmit
Removed the routes.AccountEditAvatar and routes.AccountEditUsername routes.
Fixed a data race in MemoryTopicCache.Add which could lead to the capacity limit being bypassed.
Tweaked MemoryTopicCache.AddUnsafe under the assumption that it's not going to be safe anyway, but we might as-well try in case this call is properly synchronised.
Fixed a data race in MemoryTopicCache.Remove which could lead to the length counter being decremented twice.
Tweaked the behaviour of MemoryTopicCache.RemoveUnsafe to mirror that of Remove.
Fixed a data race in MemoryUserCache.Add which could lead to the capacity limit being bypassed.
User can no longer change their usernames to blank.

Made a lot of progress on the Nox theme.
Added modified FA5 SVGs as a dependency for Nox.
Be sure to run the patcher or update script and don't forget to create a customised config/config.json file.
2018-06-17 17:28:18 +10:00
Azareal 23a686fe96 UNSTABLE: Began work on the Nox Theme.
Removed the Tempra Cursive Theme.
You can now do bulk moderation actions with Shadow.

Added:
Argon2 as a dependency.
The EmailStore.
The ReportStore.
The Copy method to *Setting.
The AddColumn method to the query builder and adapters.
The textarea setting type.
More logging to better debug issues.
The GetOffset method to the UserStore.

Removed:
Sortable from Code Climate's Analysis.
MemberCheck and memberCheck as they're obsolete now.
The obsolete url_tags setting.
The BcryptGeneratePasswordNoSalt function.
Some redundant fields from some of the page structs.

Revamped:
The Control Panel Setting List and Editor.

Refactored:
The password hashing logic to make it more amenable to multiple hashing algorithms.
The email portion of the Account Manager.
The Control Panel User List.
The report system.
simplePanelUserCheck and simpleUserCheck to remove the duplicated logic as the two do the exact same thing.

Fixed:
Missing slugs in the profile links in the User Manager.
A few template initialisers potentially reducing the number of odd template edge cases.
Some problems with the footer.
Custom selection colour not applying to images on Shadow.
The avatars of the bottom row of the topic list on Conflux leaking out.

Other:
Moved the startTime variable into package common and exported it.
Moved the password hashing logic from user.go to auth.go
Split common/themes.go into common/theme.go and common/theme_list.go
Replaced the SettingLabels phrase category with the more generic SettingPhrases category.
Moved a load of routes, including panel ones into the routes and panel packages.
Hid the notifications link from the Account Menu.
Moved more inline CSS into the CSS files and made things a little more flexible here and there.
Continued work on PgSQL, still a ways away.
Guests now have a default avatar like everyone else.
Tweaked some of the font sizes on Cosora to make the text look a little nicer.
Partially implemented the theme dock override logic.
Partially implemented a "symlink" like feature for theme directories.
... And a bunch of other things I might have missed.

You will need to run this update script / patcher for this commit.
Warning: This is an "unstable commit", therefore some things may be a little less stable than I'd like. For instance, the Shadow Theme is a little broken in this commit.
2018-05-27 19:36:35 +10:00
Azareal ba98b0c952 Added the registration logs.
Tweaked the h1 CSS for the Control Panel in Cosora.
Improved the support for autocompleters.
Reverted some of the int64s back to ints.
The per-request user variable should now have the IPs for guests.
Removed some obsolete statements.
Removed a couple obsolete phrases.
Moved a couple more accumulator initializers out of the store initializers.
Refactored one of the report statements to use a store instead of a statement.
Created the menuhead CSS class.
Added the GetOffset method to the LogStore interface and refactored the mod and admin log pages to use it.
Changed the labels on the adminlog and modlog pages to make them more understandable.
Removed the generated templates from Git.

Run the patcher / update script to get the new table.
2018-05-16 20:46:14 +10:00
Azareal 9075798128 Alerts are now rendered via a client side transpiled template rather than being hard-coded.
Tweaked some bits to make them more 32-bit friendly for GopherJS, but this might not be necessary now.
Added notice.html
Added an alerts package to fix the import cycles, more things may be moved here soon.
Saved a few lines of accumulator code in a few stores.
Moved the AccountEditCriticalSubmit, AccountEditAvatar, AccountEditAvatarSubmit, AccountEditUsername, and AccountEditUsernameSubmit routes into the routes package.
Added a QueryRow method to AccSelectBuilder.
Tweaked the indentation in the generated templates.
Simplified the template render in the AccountEditUsernameSubmit route into a redirect back to the previous page.

Run the update script / patcher to replace the route names in the viewchunks table.
2018-05-14 18:56:56 +10:00
Azareal f102c97606 The patcher can now update the schema for MySQL.
Added the adapter config value, to be used in the main system soon.
Commented out the debug code for the menus.

We're pushing some schema changes in this commit, so we'll see how well the patcher goes and whether it'll need to be patched.
2018-04-22 15:27:04 +01:00
Azareal b32e8d6c21 You can now disable slugs.
Split ops.log into ops.log and requests.log
Moved the logs into the logs directory.
Moved more password validation logic in the registration route into WeakPassword().
Tweaked the WeakPassword algorithm to cover more cases and to reduce the number of false positives.
Fixed the error grammer in WeakPassword now that the linter isn't bothering me about that anymore.
Fixed BuildGuildURL().
Removed some commented logging logic.
Bad requests are no longer logged to the console.
All bad routes are logged now.
We now track the instance uptime on the Control Panel Debug Page.
Added the executables for Linux to the .gitignore file.
Added GopherJS as a dependency.

Began work on transpiling WeakPassword to JavaScript in use in the client-side logic.
2018-04-03 05:34:07 +01:00
Azareal b20e295375 Changed the defaultest default theme to Cosora... For now.
You need the ViewIPs permission to view IPs on logs now.
Added the leftOfNav and rightOfNav docks, more on this soon.
Added a max length for topic titles.
Added a max length for usernames.
Added none as a possible language for the language view counter.
We began adding the notice phrases.
Fixed a misnamed phrase which was spitting out placeholder text.
Localised unknown for the human language phrases.
Moved routeForums to routes.ForumList.
Moved routeRobotsTxt to routes.RobotsTxt.
Started tracking the views for routes.RobotsTxt.
Moved routeSitemapXml to routes.SitemapXml.
Started tracking the views for routes.SitemapXml.
Changed the fallback theme to Cosora.
Fixed changing the default theme to Cosora or Tempra Conflux.
Removed some redundant type definitions in template init.
Return ErrNoTitle instead of ErrNoBody when trying to create a topic with no title.
Moved some in-progress routes and helper functions for handling search engine crawlers to routes/api.go
2018-03-17 08:16:43 +00:00
Azareal be47066770 General:
Added better pagination to /forum/{id}
Fixed the ordering of the elements in the pagesets.
Added hooks to the task system.
Reduced the amount of boilerplate for RunVhook and RunVhookNoReturn.
Added RunVhookNeedHook to allow plugin developers to add custom content types to the report system.
Added BulkGetMap to the DefaultPollStore.
Renamed the MultiServer config setting to ServerCount and made it an integer.
Don't cache topic lists for empty groups.
Don't load replies for topics without any.

Tempra Simple:
Added chart and submenu CSS.
Improved the input and select CSS.
Fixed the pageset CSS.
Fixed the poll input CSS.

Tempra Conflux:
Fixed the quick reply button CSS.
Hid the avatar in the quick reply area.
Added the poll CSS.
Fixed the pageset CSS.

Shadow:
Fixed the time range selector CSS.
Fixed the poll input CSS.
2018-02-15 13:15:27 +00:00
Azareal 59754b489d Block requests without known hosts for security reasons.
We now track stats for Uptimebot.
We now track stats for malformed requests.
Added json as a valid column type.

Continued work on poll posts.
Added the polls table.
Added the polls_voters table.
Added the poll column to the topics table.
Changed the type of the poll column on the replies table from boolean to int.
2018-01-25 04:57:33 +00:00
Azareal bdbd80319f Made some minor tweaks.
The DefaultRoute parameter in the configuration struct is now a string.
Fixed a few mismatched HTML tags.
Added the profile_comments_row template.
Added more alt attributes for images and associated title=""s in a few places.
Connections should now be closed properly.
Custom Pages no longer have a .html file extension in their URLs.
We now track route views for the default route and the custom page route.

Began work on cleaning up the profiles on Cosora.
2017-12-22 03:32:23 +00:00
Azareal 10a0c62823 The Cosora Theme is almost complete and is being rolled out on the site to demo it.
We now track the views on a per-route basis. We have plans for an admin UI for this, global views, etc. which will come in a future commit.

The local error JSON is now properly formed.
Fixed an outdated line in topic.go which was using the old cache system.
We now use fuzzy dates for relative times between three months ago and a year ago.
Added the super admin middleware and the associated tests.
Added the route column to the viewchunks table.
Added more alt attributes to images.
Added a few missing ARIA attributes.

Began refactoring the route generator to use text/template instead of generating everything procedurally.
Began work on per-topic view counts.
2017-12-19 03:53:13 +00:00