Writing your own custom forum
It is a rare treat to see a custom-written forum system on a website, seeing as the number of free forum systems you can download today is approaching infinity. They are mostly written in PHP (MyBB, phpBB, IPB, SimpleMachines, Vanilla, to name only a few of the most popular) which is logical since PHP is widely supported, free, runs on free platforms and uses a free database program - just about anyone can use these scripts to start their own community, so why would anyone want to write a forum from scratch? Well probably because:
- It's a challenge, like assembling a matchstick Eiffel Tower
- It's a fun project that you can learn a lot from
- You get to be different from other sites (people who appreciate forum styles are going to be delighted)
- You can make a forum with less clutter, and more of your own features - the ones you've always wanted
- You can write it in a language that there are very little forum systems written in (e.g. Ruby)
- You need to spend time making every feature you want
- You have to maintain the code, fixing any bugs or issues that may be discovered (or else)
- Your system won't be compatible with plugins or styles other users may be using
Basic prerequisites
You'll need to know HTML and CSS perfectly, with a decent knowledge of JavaScript (preferrably). You'll also need a good knowledge of a server-side scripting language like PHP, Perl or Python, as well as an understanding of SQL to interact with the database of your choice (like MySQL).
Before you start, you're going to want to give a little thought about what your forum will look like, and what features it will have apart from the standard view, edit, delete and post new topics and posts, login and register, user profiles, a user and staff control panel. It helps if you jot these ideas down, even if they're just small and seemingly obvious details. Also, you should make a test.html page of your forum's layout - making the design first should be a big help seeing as you can start using that right away, and it should let you have an idea of what is going to go where.
Devise a good database schema
By arranging where you store what data (in which tables), you can use queries more efficiently and get a faster resulting forum due to less processing and less queries. For example, having a user's postcount and id of their last post in the same table as the rest of the user information (username, password, ...) is a good thing as it will save you querying for it. The same applies for the boards your forum will have - make sure to have the last topic id and last poster id in that table for quicker processing. So don't be afraid to sacrifice some space in the database (you could just as well fetch the post count for each user with COUNT(*), but this method is faster because it doesn't require any special work on the database's side. Also, keep your table names lowercase and similar in naming style, for example don't have users_subscriptions and usercomments, make the table and field names logical and easy to remember. You should bear this in mind, and create at least a few simple tables in your database that you can always add new fields to during development.
Lay down conventions, and follow them
It's no good if you use a different style in every other script. Things like how you name functions, variables and how you position your code is important, and it will make it easier for you to read and edit your own code if you follow these style conventions. Also, there may be certain language-specific conventions that you may want to define early in the development process. In PHP, for example, it may become annoying if you use && in one place, and then "and" and "AND" somewhere else. Even seemingly small things such how you position your brackets, quotes, whether you write function(x) or function( x ) will matter when you're looking for a bug. Also, if you have a designed a specific way of doing something - for example, verifying whether the thread exists by use of a function verify_thread_id() - make sure any other developers that you may be working with know about it.
Core systems
Whatever language you use, you will need to know that you're about to write a lot of code, and it's going to get real messy real fast if you don't have one global way of doing things. Don't settle for quick hacks, make sure everything you make preforms as expected, and will preform as expected. One way of making things as easy as possible for yourself is by making some basic classes and functions that will help you in development.
The most obvious is a database abstraction class, a class that describes an object that will preform database queries and stuff, regardless of the database. In PHP for instance, common database choices are mysql and mysqli, though both use different functions for connecting, querying and so forth, mysql_x vs mysqli_x. Even if you know you're going to be using only one database program for your site, you still don't want to have to keep track of all those connection variables, querycounts and such things every time you query. That will lead to messy code - instead, write a class to manage it. In PHP, I had a database class that creates the object $db. $db->connect(); connects to the mysql server, and even selectes the correct database. $db->close(); closes the connection. Running a query is as simple as $db->query("SELECT * FROM foo"); - not only does that run the query, but it also updates the query count, and it can even differentiate queries that read from the server and those that write to the database, and based on that use different SQL servers if necessary (master/slave). So this gets to be an enormous benefit. In fact, even if you choose for some reason (beats me) to use a flat-file format instead of a database, you be in an even more dire need of a set of standard functions and procedures to help you manage the stored information. Likewise, having a smart, easy to use class for the most common tasks gets to be a real time-saver that you will recognise as an outstanding benefit as you progress deeper into implementing your own forum software - or any larger web app at all.
Just as is the case with the database, you need to simplify tasks to do with permissions, templating, parsing bb code and so forth. Templating is the one and only solution to consider when making a forum: it is just plain impossible to manage code and not get broken HTML if you mix your code and your HTML (very easy mistake to make in PHP - if you're working in ASP.NET you don't need one; if you're using a web framework you most likely won't have to implement one youself). I've described how a templating system works in my first article, Templating with PHP, so check that out if you haven't already.
A permission system is different. What this will do is provide some functions to use when checking whether a user is allowed to perform a particular action. You'll find it a lot easier to make than the templating system if you use a good method of implementing permissions. Basically you'll need some kind of identifier (like "canmaketopic") and a value (like "yes"). Of course though that would work, there are more efficient ways of doing this. Some forums like MyBB use what I call a "description"-based permission system, which means every usergroup has a lot of fields in the database, one for each type of permission and each field containing either "yes" or "no". Though this is fine and friendly toward the administrators if you don't have an admin panel and work directly on the database, you can do better. A better type of permission system would use only one field for each permission category (like staff permissions , general permissions , and similar). You could use yes and no separated by verical pipes, like yes|no|yes|yes|yes|no but this will prove difficult to manage and easy to get bugs in. The best solution I know of is using a "bit field" approach (bit fields are a way of "storing" several options or permissions in a single number by using powers of two and bitwise operators). I've written a separate article that goes into more detail about this, since it will be too long to discuss here - see a bitfield permission system.
A BB Code parser (BB code = bulletin board code) is a piece of code to parse bb tags into their HTML equivalents. A decent BB code parser is possibly one of the most complex pieces of code you're going to use on your forum, so if you don't feel up to the task, you should try getting a free one from the internet and modifying it to fit your needs. If you want to however, all I'm going to say about bb code parsers right now is that they work by finding tags in text, making sure they're correctly nested within one another (XHTML requirement) and that they contain valid information (the case of the url, size, colour and such tags), and then replacing them in the appropriate way with HTML.
Naturally, the more of these "modules" you make, the less you will need to repeat yourself in your code (meaning less bugs and bugs that are easier to fix than usual) and an ease-of-development factor you will love. This goes for functions too, not just classes. Why not use a function called get_user_info() to get the data for the currently browsing user? Perhaps you could make it so that the function only needs to run the actual query once, regardless of how many times you call it, so you don't waste execution time querying for the same info? In any case, you may not be able to recognise what code you will be repeating a lot, but you can always add to your functions and classes, and make the whole system more organised and thus easier to maintain and expand.
Usergroups/Ranks
These are groups your members will belong to, such as member, banned, guest, admin... you should have a table called ranks or usergroups in your database, which will contain the ID numbers, names, descriptions and permissions for each group, for easy editing and adding extra groups. You might want to have icons for each group, so make sure you've got a field in the table for that. After this, it's easy. Let every user have a rank field in their row, and updating the id of the rank to something else makes them a member of that group, and they use that group's opening and closing HTML tag (if you want the names of different usergroups to be surrounded by different tags) and permissions and stuff.
Registration and Login
This is one of the first features you should make after making the basic things such as users, usergroups and topic/forum viewing scripts. With login, don't just set cookies with the user's ID and password. Also have a session table in the database that will list all users currently using the site and their unique session IDs (these could be md5 digests of the user's browser and IP - things that won't change during a session) so that even if someone was to steal the cookies of a user, not only would they not know the password (because it was hashed by md5 or another message-digest algorithm) but they could not create the cookies for themselves and log into your forums, because the session in the database would notice that there had been a change, and would require the user to log in.
The Admin Panel
You will probably feel that making the admin control panel is the most boring part of your project. It won't ever be seen by anyone except several users (if your security is right) so you'll feel a funny nervous feeling in your arms and you'll try and make the panel as quickly as possible, settling for hacky solutions. That's ok. Having any kind of admin panel is better than nothing in any case, and since it won't ever be used by more than several individuals, if the interface isn't your finest work ever, it doesn't matter. Everything you make should work, but you don't need to validate what is inputted as much as everywhere else.
Moderator functions
It may not seem like it, but having a perfectly working system of moderator functions (close/open, stick, delete, edit) is a very important thing. Very soon after your visitors start registering, you will need to have these functions working for your own use, and for any moderators you choose to assign. If they don't work perfectly, your forum has the potential of getting out of hand. Just sit down and make these right after you make the edit/delete features, and you'll be fine. And of course, make sure your forum and topic displaying scripts don't allow posting in locked topics, and correctly display sticky threads.
Three important things
First: make sure unicode is supported, and you use a unicode transformation format (UTF, like utf-8) for encoding. Anyone should be able to enter Chinese or Japanese characters, symbols such as the trademark symbol, and accented characters of all sorts. Getting little black diamonds with question marks in them is seriously annoying. So is getting incorrect characters due to encoding mix-ups.
Second: Remove backslashes from escaped quotes before you output them. You should be escaping strings as a part of stopping SQL injection, but people often forget to do this. (unless you're using a different language than PHP :P)
Third: Solve any special HTML entity problems you may have. If editing a post escapes the ampersand in " again, so you get ", fix it soon. It's very annoying. Anyway, you should only be sanitizing HTML once, either before you insert into the database (easier to do) or every time after you fetch it from the database.
Respect general forum conceptions
There are some things your users will expect from your forum, like that it will be split into sections dealing with boards, topics and messages (alternatively forums, threads and posts), and that it will resemble what their conception of a forum is. Some users like myself find using radically different forum systems such as Vanilla a difficult, confusing experience until they get used to it - your forum will be even more challenging to use since it's the only one of it's kind, whereas hundreds of sites run Vanilla, so there are many users who are already super-familiar with it's interface. Don't take too much liberty in changing some things such as where you would expect certain features to be on the page, or you will confuse your users and your forum will lose a lot of usability. But apart from that, feel free to get creative: as far as the layout is concerned, there are many, many possible ways that your forum can display information to users, so try some out!
You can still listen to Rush, I'm talking about rushing with your coding. Some people say, "it's all about the journey, not the destination". I am not sure that's always accurate, though you should take your time when making your forum. It can be simple, but your users will know when it's rushed. Make sure everything important works seamlessly, such as forum and thread listing, and users and stuff. Use javascript to make it more responsive and quicker to use. Take your time with this, space your code out so it's readable, and document your code!! No one ever died from writing too many comments in software development. This is especially important if you're working with other people, so never skip it.
Where to start?
If you're a bit confused about where to start, start with the layout. Have one or two static pages that show what your forum will look like, and then take that and start making it dynamic. You could try taking that HTML and using a templating system to generate it dynamically. Next you can connect to the database, and make some of the things on the page dynamic, like the current date, etc. Then set up users and groups, registration and login. You should then make basic features of reading and creating posts (by this time you should be using permissions), forum listing and locked/sticky topics. After that it's all quite easy, you just have to polish up what you've got, and then start adding things like a parser for BB code, user control panel for updating personal information and avatars, and whatever else you might want. You should thoroughly test everything you make, and when you're ready start letting people sign up. There you have your own hand-made community!
And that's about it. The most important thing is to have fun, and learn from the experience. If you want to make a really good forum, you will often either get so excited that you simply have to complete some feature, and will work deep into the night. In other cases, you will still be working deep into the night working on stuff because you'll require extra time that our obligation-packed, short 12-hour days don't provide. In any case, if you're anything like me, you'll love every minute of it, but you can expect to suffer from "indie forum coder's syndrome". Some symptoms are: sore, bloodshot eyes; big dark bag-like entities just under the eyes, messy hair from lack of attention to personal looks, occasional confusion when asked questions not pertaining to web programming, stomach aches from insufficient quantities of food and too much coffee, hypovitaminosis D.
In any case, I will probably add to this article and ammend it. If you have any questions or comments, please drop me a line or two. Thanks for reading!
Jul 27 2010 @ 16:04:19
I remembered that you had something about custom something for websites on here, so I decided to look it up. I started scanning this, and saw the 'Don't rush!' section. The first sentence made me laugh.
I might try to use this, if I can.
I might try to use this, if I can.
Jun 14 2008 @ 16:02:16
... Eek sorry
Cool :D
Cool :D
