Commit Graph

27 Commits

Author SHA1 Message Date
Azareal
459d745cb1 initial perf anaytics
add tasks to debug page
ignore .git on debug page for speed

add perfchunks table

Renamed phrases (changed statistics to stats):
panel_menu_stats
panel_menu_stats_posts
panel_menu_stats_topics
panel_menu_stats_forums
panel_menu_stats_routes
panel_menu_stats_agents
panel_menu_stats_systems
panel_menu_stats_languages
panel_menu_stats_referrers
panel_menu_stats_memory
panel_menu_stats_active_memory
panel_menu_stats_perf
panel_stats_views_head_suffix
panel_stats_user_agents_head
panel_stats_forums_head
panel_stats_languages_head
panel_stats_post_counts_head
panel_stats_referrers_head
panel_stats_routes_head
panel_stats_operating_systems_head
panel_stats_topic_counts_head
panel_stats_requests_head
panel_stats_memory_head
panel_stats_active_memory_head
panel_stats_spam_hide
panel_stats_spam_show
panel_stats_memory_type_total
panel_stats_memory_type_stack
panel_stats_memory_type_heap
panel_stats_time_range_one_year
panel_stats_time_range_three_months
panel_stats_time_range_one_month
panel_stats_time_range_one_week
panel_stats_time_range_two_days
panel_stats_time_range_one_day
panel_stats_time_range_twelve_hours
panel_stats_time_range_six_hours
panel_stats_post_counts_chart_aria
panel_stats_topic_counts_chart_aria
panel_stats_requests_chart_aria
panel_stats_memory_chart_aria
panel_stats_details_head
panel_stats_post_counts_table_aria
panel_stats_topic_counts_table_aria
panel_stats_route_views_table_aria
panel_stats_requests_table_aria
panel_stats_memory_table_aria
panel_stats_views_suffix
panel_stats_posts_suffix
panel_stats_topics_suffix
panel_stats_user_agents_no_user_agents
panel_stats_forums_no_forums
panel_stats_languages_no_languages
panel_stats_post_counts_no_post_counts
panel_stats_referrers_no_referrers
panel_stats_routes_no_routes
panel_stats_operating_systems_no_operating_systems
panel_stats_memory_no_memory

Added phrases:
panel_debug_tasks
panel_debug_tasks_half_second
panel_debug_tasks_second
panel_debug_tasks_fifteen_minute
panel_debug_tasks_hour
panel_debug_tasks_shutdown
panel_stats_perf_head
panel_stats_perf_low
panel_stats_perf_high
panel_stats_perf_avg

You will need to run the updater / patcher for this commit.
2020-02-23 19:08:47 +10:00
Azareal
3d1fd807d3 add RemoveIndex adapter method
add AddKey installer method
2020-02-21 09:55:45 +10:00
Azareal
84cdc097e7 Add UseConvosOnlyWithMod group permission.
Add UseConvosOnlyWithMod phrase.
Add initial convo tests.
Bar access to ConvosCreate route if we don't have the needed perms.
2020-02-11 13:13:38 +10:00
Azareal
0c1d6f0516 Add CreateProfileReply and AutoEmbed group permissions.
Log profile reply deletions in the moderator log.
Split the global permissions in the UI to make them easier to manage.
Experiment with showing group ID in group edit header.
Avoid loading groups multiple times for the same profile reply.

Initialise disabled IP log points to empty string rather than 0.

Add CreateProfileReply perm phrase.
Add AutoEmbed perm phrase.
Add panel_group_mod_permissions phrase.
Add panel_logs_mod_action_profile_reply_delete phrase.
2020-02-04 21:47:03 +10:00
Azareal
6935637867 Cascade delete attachments properly.
Cascade delete replied to topic events for replies properly.
Cascade delete likes on topic posts properly.
Cascade delete replies and their children properly.
Recalculate user stats properly when items are deleted.
Users can now unlike topic opening posts.
Add a recalculator to fix abnormalities across upgrades.
Try fixing a last_ip daily update bug.

Add Existable interface.
Add Delete method to LikeStore.
Add Each, Exists, Create, CountUser, CountMegaUser and CountBigUser methods to ReplyStore.
Add CountUser, CountMegaUser, CountBigUser methods to TopicStore.
Add Each method to UserStore.
Add Add, Delete and DeleteResource methods to SubscriptionStore.
Add Delete, DeleteByParams, DeleteByParamsExtra and AidsByParamsExtra methods to ActivityStream.
Add Exists method to ProfileReplyStore.

Add DropColumn, RenameColumn and ChangeColumn to the database adapters.

Shorten ipaddress column names to ip.
- topics table.
- replies table
- users_replies table.
- polls_votes table.
Add extra column to activity_stream table.
Fix an issue upgrading sites to MariaDB 10.3 from older versions of Gosora. Please report any other issues you find.

You need to run the updater / patcher for this commit.
2020-01-31 17:22:08 +10:00
Azareal
45a3065db1 Use json marshal for the group permissions in querygen.
Add an omitempty json tag to the Perms struct.
Take the opportunity to shorten some things.
2019-10-06 15:32:08 +10:00
Azareal
5705252029 Add UseConvos permission.
Use UseConvos permission instead of ban flags in convo perm checks.
Stop users without the UseConvos permission from editing convo replies, although they can still delete them for privacy reasons.

Shorten some things and reduce the amount of boilerplate.
Add a few misc parser test cases.
Fix footer and tweak indentation.
2019-10-06 10:34:09 +10:00
Azareal
ef4a88c43f Fix an index related bug in the patcher.
Fix AS in columns in left joins.
Remove a redundant AS in simple count.
Remove a couple of unused session parameters.
Shorten a few session bits.
2019-07-29 11:31:19 +10:00
Azareal
199a841bc3 Add the UploadAvatars permission.
Let users revoke their own avatars, work in progress.
Reduce boilerplate in debug.go

Add the UploadAvatars phrase.
Add the account_penalties phrase.
Add the account_menu_penalties phrase.
Add the account_avatar_revoke_button phrase.

More Vue and Conversations Work.
2019-06-09 13:21:48 +10:00
Azareal
da86280e1e bleh schema 2019-05-28 10:47:30 +10:00
Azareal
cc1d0f089a Client templates can now be overriden like every other template.
The client templates load earlier now for a smoother user experience.
Added a setting for setting a Google Site Verification meta tag without editing header.html
Added support for favicons. /static/favicon.ico will be mapped to favicon.ico, if it exists.
The parent forum is now visible on the topics list for Nox.

Language headers which contain the wildcard character are no longer considered unknowns.
Meta descriptions and open graph descriptions are no longer emitted for logged in users.
Slimmed down topics_topic slightly for Nox.
Pre-parsed widgets are now minified.
Stale WebSockets connections should be cleaned up far quicker now.
Template generation is now logged separately.
Commented out some obsolete template logic.
Marked a few template generator fields as unexported.

Fixed the styling for the ban page in the profile for Nox.
Fixed the styling for colline for Cosora and Tempra Simple.
Fixed the sidebar overflowing outside of the box on Nox.
Fixed the meta description text overflowing the box in the Setting Manager on Nox.
Fixed excessive padding in the Page Manager.
Fixed a few missing border on the profiles for Tempra Simple.
Fixed the sidebar appearing in places it shouldn't on Tempra Simple.
Fixed the status code emitted by NotFoundJS
Fixed a bug where Gosora kept falling back to interpreted templates.
Fixed a bug where WebSockets connections weren't getting closed properly if the user cache overflowed.
Fixed a bug where WebSocket connections weren't getting initialised for guests.
Fixed a bug where template overrides weren't always getting applied.
Fixed a bug where root template overrides weren't always getting applied.

Added the google_site_verify setting.
Added the google_site_verify phrase.

You will need to run the patcher or updater for this commit.
2019-02-28 17:28:17 +10:00
Azareal
5db5bc0c7e Reply attachments can be managed too now.
Added eight database indices.
Fixed a bug where the second tick wouldn't fire.
Tweaked the .topic_forum in Nox by a pixel.
Replaced some _installer strings with blank strings for consistency with the builders.
Greatly reduced the number of allocations in the user agent parser.
Added ampersand entities in more attachment URLs to avoid accidental mangling.
.edit_source is now hidden for guests.
Guest noavatars are now pre-calculated to reduce the number of allocations.
Lazily initialised a couple of maps in ViewTopic to reduce the number of unnecessary allocations slightly.

Added the unsafe BytesToString function. Please don't use this, if you don't have to.
Added the AddIndex method to the adapter and associated components.
Added the /reply/attach/add/submit/ route.
Added the /reply/attach/remove/submit/ route.
Added the topic_alt_userinfo template.
Replaced Attachments.MiniTopicGet with MiniGetList.
Added Attachments.BulkMiniGetList.

Added a quick test for ReplyStore.Create.
Added BenchmarkPopulateTopicWithRouter.
Added BenchmarkTopicAdminFullPageRouteParallelWithRouter.
Added BenchmarkTopicGuestFullPageRouteParallelWithRouter.

You will need to run the updater or patcher for this commit.
2018-12-31 19:03:49 +10:00
Azareal
2a5ab2969c Added Caire and pkg/errors as dependencies.
Made the initialisers and the task runner in main.go easier to debug.

Added form_button_row to a few forms.
Bumped up the attachment image size for Cosora.
Hid the formlabels for the setting editor for Cosora to make it cleaner.
Revamped the account manager with the same CSS as in the Control Panel for Nox.
Started adding the_form to more forms.
Removed the account_emails CSS class from the email editor.
Continued tweaking the Control Panel in Nox to make it look nicer.
Tweaked some of the headers in the Nox Theme.
Added the Create Topic and Moderate options to the topic list on the Nox Theme, although the bulk moderation tools aren't available yet.
Tweaked the padding and sticky shades on the topics on the topic list on the Nox Theme.
Closed topics are now somewhat styled on the topic list on the Nox Theme.
Continued work on the topic pages for Nox.

Renamed Admin Approval to Staff Approval in the English Language Pack.
Added more phrases for the Group Manager and Panel Menu in spots I overlooked.

Began work on the dyntmpl template function.
Be sure to run the patcher / update script to get the new setting.
2018-07-13 21:27:58 +10:00
Azareal
f8f46b3c48 Added support for two-factor authentication.
Added the Account Dashboard and merged a few account views into it.
BREAKING CHANGE: We now use config/config.json instead of config/config.go, be sure to setup one of these files, you can config_default.json as an example of what a config.json should look like. If you don't have an existing installation, you can just rely on the installer to do this for you.

CSS Changes (does not include Nox Theme):
Sidebar should no longer show up in the account manager in some odd situations or themes.
Made a few CSS rules more generic.
Forms have a new look in Cosora now.

Config Changes:
Removed the DefaultRoute config field.
Added the DefaultPath config field.
Added the MaxRequestSizeStr config field to make it easier for users to input custom max request sizes without having to use a calculator or figure out how many bytes there are in a megabyte.
Removed the CacheTopicUser config field.
Added the UserCache config field.
Added the TopicCache config field

Phrases:
Removed ten english phrases.
Added 21 english phrases.
Changed eleven english phrases.
Removed some duplicate indices in the english phrase pack.

Removed some old benchmark code.
Tweaked some things to make the linter happy.
Added comments for all the MemoryUserCache and MemoryTopicCache methods.
Added a comment for the null caches, consult the other caches for further information on the methods.
Added a client-side check to make sure the user doesn't upload too much data in a single post. The server already did this, but it might be a while before feedback arrives from it.
Simplified a lot of the control panel route code with the buildBasePage function.
Renamed /user/edit/critical/ to /user/edit/password/
Renamed /user/edit/critical/submit/ to /user/edit/password/submit/
Made some small improvements to SEO with a couple of meta tags.
Renamed some of the control panel templates so that they use _ instead of -.
Fixed a bug where notices were being moved to the wrong place in some areas in Cosora.
Added the writeJsonError function to help abstract writing json errors.
Moved routePanelUsers to panel.Users
Moved routePanelUsersEdit to panel.UsersEdit
Moved routePanelUsersEditSubmit to panel.UsersEditSubmit
Renamed routes.AccountEditCritical to routes.AccountEditPassword
Renamed routes.AccountEditCriticalSubmit to routes.AccountEditPasswordSubmit
Removed the routes.AccountEditAvatar and routes.AccountEditUsername routes.
Fixed a data race in MemoryTopicCache.Add which could lead to the capacity limit being bypassed.
Tweaked MemoryTopicCache.AddUnsafe under the assumption that it's not going to be safe anyway, but we might as-well try in case this call is properly synchronised.
Fixed a data race in MemoryTopicCache.Remove which could lead to the length counter being decremented twice.
Tweaked the behaviour of MemoryTopicCache.RemoveUnsafe to mirror that of Remove.
Fixed a data race in MemoryUserCache.Add which could lead to the capacity limit being bypassed.
User can no longer change their usernames to blank.

Made a lot of progress on the Nox theme.
Added modified FA5 SVGs as a dependency for Nox.
Be sure to run the patcher or update script and don't forget to create a customised config/config.json file.
2018-06-17 17:28:18 +10:00
Azareal
23a686fe96 UNSTABLE: Began work on the Nox Theme.
Removed the Tempra Cursive Theme.
You can now do bulk moderation actions with Shadow.

Added:
Argon2 as a dependency.
The EmailStore.
The ReportStore.
The Copy method to *Setting.
The AddColumn method to the query builder and adapters.
The textarea setting type.
More logging to better debug issues.
The GetOffset method to the UserStore.

Removed:
Sortable from Code Climate's Analysis.
MemberCheck and memberCheck as they're obsolete now.
The obsolete url_tags setting.
The BcryptGeneratePasswordNoSalt function.
Some redundant fields from some of the page structs.

Revamped:
The Control Panel Setting List and Editor.

Refactored:
The password hashing logic to make it more amenable to multiple hashing algorithms.
The email portion of the Account Manager.
The Control Panel User List.
The report system.
simplePanelUserCheck and simpleUserCheck to remove the duplicated logic as the two do the exact same thing.

Fixed:
Missing slugs in the profile links in the User Manager.
A few template initialisers potentially reducing the number of odd template edge cases.
Some problems with the footer.
Custom selection colour not applying to images on Shadow.
The avatars of the bottom row of the topic list on Conflux leaking out.

Other:
Moved the startTime variable into package common and exported it.
Moved the password hashing logic from user.go to auth.go
Split common/themes.go into common/theme.go and common/theme_list.go
Replaced the SettingLabels phrase category with the more generic SettingPhrases category.
Moved a load of routes, including panel ones into the routes and panel packages.
Hid the notifications link from the Account Menu.
Moved more inline CSS into the CSS files and made things a little more flexible here and there.
Continued work on PgSQL, still a ways away.
Guests now have a default avatar like everyone else.
Tweaked some of the font sizes on Cosora to make the text look a little nicer.
Partially implemented the theme dock override logic.
Partially implemented a "symlink" like feature for theme directories.
... And a bunch of other things I might have missed.

You will need to run this update script / patcher for this commit.
Warning: This is an "unstable commit", therefore some things may be a little less stable than I'd like. For instance, the Shadow Theme is a little broken in this commit.
2018-05-27 19:36:35 +10:00
Azareal
d0318191c9 Added the menu manager and menu item editor.
Refactored the menu system.
Updated the README and revamped it a tad to make it easier to understand. Also, added manual instructions for patching.
Revamped the update scripts, especially on Windows.
Merged the CSS and Tmpl phrase namespaces.
Added lastSchema to .gitignore
Added DropTable to the database adapters.
Implemented DbVersion in the PgSQL Adapter.
Swapped out the checkboxes for cleaner looking yes-no dropdowns.
Began revamping small bits of the user logic.

We now open to contributions, just open a pull request and sign the CLA.
Schema has been updated, run the patcher or update script.
2018-05-11 15:41:51 +10:00
Azareal
15420d4d89 The patcher (schema updating part of the updater) finally works, yay.
Partially rewrote the forum permissions system to make it more stable.

Moved config.go into it's own package in /config/
Removed Go Git as a dependency.
Tweaked the GopherJS pulling.
Fixed inserts where all the columns have default values.
Reverted a silly tweak I made thinking that mOrder == order.
Removed the /common/ hack from the patcher.
Fixed a bug where the forum creator would ignore the visiblity value you provided.
The tests now work again.
Swapped a misplaced fmt.Println with a fmt.Printf.
Fixed a bug in the installer where all the table logs would be on one line in the console.
Added more logging to the installer.
2018-04-23 22:08:31 +01:00
Azareal
dbf2b8606e All the menu items are dynamic now, all that needs to be done now is the Control Panel part of the Menu Manager.
Travis should now build the patcher to see if that works.
Merged menu.html into header.html
Removed a redundant SetAdapter call in mysql.go.
Fixed a bug where inserting will sometimes strip the spaces in strings.
Bumped the minimum Go Version in schema.json to 1.10
Bumped the DBVersion to 1, so that we don't install these patches twice.

The patcher was botched in the previous commit, drop menus and menu_items and run it again
2018-04-23 09:38:25 +01:00
Azareal
10f4c59cb5 Fixed the Go Version in the Travis file.
Added the Go and Database versions to the Control Panel Debug Page.
Renamed common.TopicsPage to common.TopicListPage.
Renamed *HeaderVars to *Header.
Added the Paginator struct and refactored the code to use it.
io.Writers are now used instead of http.ResponseWriters in transpiled templates for greater flexibility.
Added the alert, menu_alerts, and menu_item templates.
Added support for more integer types in the arithmetic functions for the transpiled templates.
Exported AccSelectBuilder.
Added an Each method to AccSelectBuilder.
Added column quoting to the order by portions of queries for the MySQL Adapter.

Began work on the client side rendering of alerts.
Began work on the Menu Manager and associated functionality.
2018-04-22 13:33:56 +01:00
Azareal
b20e295375 Changed the defaultest default theme to Cosora... For now.
You need the ViewIPs permission to view IPs on logs now.
Added the leftOfNav and rightOfNav docks, more on this soon.
Added a max length for topic titles.
Added a max length for usernames.
Added none as a possible language for the language view counter.
We began adding the notice phrases.
Fixed a misnamed phrase which was spitting out placeholder text.
Localised unknown for the human language phrases.
Moved routeForums to routes.ForumList.
Moved routeRobotsTxt to routes.RobotsTxt.
Started tracking the views for routes.RobotsTxt.
Moved routeSitemapXml to routes.SitemapXml.
Started tracking the views for routes.SitemapXml.
Changed the fallback theme to Cosora.
Fixed changing the default theme to Cosora or Tempra Conflux.
Removed some redundant type definitions in template init.
Return ErrNoTitle instead of ErrNoBody when trying to create a topic with no title.
Moved some in-progress routes and helper functions for handling search engine crawlers to routes/api.go
2018-03-17 08:16:43 +00:00
Azareal
2436d9a6fa You can now bulk move topics from one forum to another.
We now track the User Agent for Cloudflare Always On.
2018-01-15 08:24:18 +00:00
Azareal
e919812ee2 Added support for meta descriptions. You can configure this from the Control Panel's Setting Manager.
Added tests for forum creation.
2017-12-25 06:12:19 +00:00
Azareal
0e9cebfa47 Overhauled the widget system. You can now specify more complex logic for where a widget can show up and you can now place widgets in the footer.
Added fsnotify as a dependency, we'll be using it, Rez, and Riot (maybe) soon.
Removed the intercept_build_widgets vhook.
You can now pass an ID to BlankReply() for mocking replies or whatever else you want to do with this.
Added the dock template function for widget docks.
Added the logf internal function to the template transpiler and did some minor clean-up.
Removed the Sidebars property from theme.json and added the more general Docks one.
Improved the footer and the associated CSS.
Added the about widget, previously the AboutSegment feature.
Removed the about_segment_title and about_segment_body settings.
Gosora now exits when it receives the appropriate OS signal.
Refactored the modlogs route.

More progress on the theme in the Control Panel.
2017-11-29 02:34:02 +00:00
Azareal
381ce3083a Added the AboutSegment feature, you can see this in use on Cosora, it's a little raw right now, but I'm planning to polish it in the next commit.
Refactored the code to use switches instead of if blocks in some places.
Refactored the Dashboard to make it easier to add icons to it like I did with Cosora.
You can now use maps in transpiled templates.
Made progress on Cosora's footer.
Swapped out the ThemeName property in the HeaderVars struct for a more general and flexible Theme property.
Added the colstack CSS class to make it easier to style the layouts for the Control Panel and profile.
Renamed the FStore variable to Forums.
Renamed the Fpstore variable to FPStore.
Renamed the Gstore variable to Groups.
Split the MemoryTopicStore into DefaultTopicStore and MemoryTopicCache.
Split the MemoryUserStore into DefaultUserStore and MemoryUserCache.
Removed the NullUserStore, SQLUserStore, and SQLTopicStore.
Added the NullTopicCache and NullUserCache.
Moved the Reload method out of the TopicCache interface and into the TopicStore one.
Moved the Reload method out of the UserCache interface and into the UserStore one.
Added the SetCache and GetCache methods to the TopicStore and UserStore.
Added the BypassGetAll method to the WordFilterMap type.
Renamed routePanelSetting to routePanelSettingEdit.
Renamed routePanelSettingEdit to routePanelSettingEditSubmit.
Moved the page titles into the english language pack.
Split main() into main and afterDBInit to avoid code duplication in general_test.go
Added the ReqIsJson method so that we don't have to sniff the headers every time.
Added the LogStore interface.
Added the SQLModLogStore and the SQLAdminLogStore.
Refactored the phrase system to use getPhrasePlaceholder instead of hard-coding the string to return in a bunch of functions.
Removed a redundant rank check.
Added the GuildStore to plugin_guilds.
Added the about_segment_title and about_segment_body settings.
Refactored the setting system to use predefined errors to make it easier for an upstream caller to filter out sensitive error messages as opposed to safe errors.
Added the BypassGetAll method to the SettingMap type.
Added the Update method to the SettingMap type.
BulkGet is now exposed via the MemoryUserCache.
Refactored more logs in the template transpiler to reduce the amount of indentation.
Refactored the tests to take up fewer lines.
Further improved the Cosora theme's colours, padding, and profiles.
Added styling for the Control Panel Dashboard to the Cosora Theme.
Reduced the amount of code duplication in the installer query generator and opened the door to certain types of auto-migrations.
Refactored the Control Panel Dashboard to reduce the amount of code duplication.
Refactored the modlog route to reduce the amount of code duplication and string concatenation.
2017-11-23 05:37:08 +00:00
Azareal
569b424ac8 Did a bit of refactoring on the template system.
Refactored the ReplyStore.
Added the Create method to the ProfileReplyStore.
Fixed a potential issue with the topics and replies generated by the installer.
Refactored the accumulator.
Refactored global.js

Added more tests.
2017-11-06 16:24:45 +00:00
Azareal
fdb6304e32 Mostly a mid-way technical commit to make reading diffs easier on me.
Split the relativeTime function into relativeTime and relativeTimeFromString.
Refactored the installer.
Made a lot of progress with PgSQL and MSSQL support, mostly MSSQL.
Made a lot of progress on the automated testing suite.
Added eqcss to the file extension list.
..depressed..
Fixed various bugs here and there.
gen_mysql.go is now properly excluded when the pgsql or mssql build tag is passed.
The BBCode plugin is now always initialised prior to it's test.
We've begun transitioning towards deserializing database times directly into time.Times rather than doing it every time on the spot.
Added more dev flags.
The tests now make use of a test database rather than the production database.
2017-10-14 08:39:22 +01:00
Azareal
e30707bc17 Refactored the installer's code. We're moving towards supporting multiple database engines.
More progress has been made with PostgreSQL, but it's still incomplete and experimental.
The items on the Social Group menu bar are no longer decorative and now let you jump from page to page.
Social Group permissions are now fully functional. More features to come like privacy levels, secondary board permissions, etc. for Social Groups.

One of our dependencies added a dependecy, so we've added that dependency as a dependency.
Tweaked the ForumStore to better conform to the tests.
Fixed an issue with the generated file comment stopping the build tags from being read by the compiler for a few files.
Fixed the test system.
Renamed the route_create_topic handler to route_topic_create_submit
Moved the user table into the query generator.
Fixed a bug in MySQL where it doesn't allow unique keys longer than 180 characters.
Fixed a race condition in the forum deletion handler.
Fixed a crash bug where Gosora crashes when [rand]0[/rand] is passed to the BBCode parser.
Fixed a bug with superadmins not being able to see all the forums they should be able to.
Fixed a bug in simple_forum_session_check where it assumes every request has an error.

Tests:
Added 8 ForumStore tests.
Added 4 Auth tests.
Added 14 bbcode_full_parse tests.
Added 4 bbcode_regex_parse tests.
Fixed a bug in one of the bbcode_full_parse tests.

Apparently, routes.go wasn't commited in the previous commit o_o
2017-07-12 12:05:18 +01:00