Go in Server Development

Introduction

Conquer Online server development usually consists of large projects and complex server designs; and unfortunately, choosing a best-fit programming language isn't always straightforward with projects of this scale. All programming languages come with their advantages and disadvantages, and Go is no exception to this. In this article, I talk about my decision to use Go for Chimera and how Go had the least disadvantages in comparison to other languages.

What is the purpose of Go?

In case you're not aware - Google first developed Go to help replace aging C++ systems which were becoming more difficult to maintain and compile in a timely manor. Go was designed by engineers whom were frustrated with the features and environments of existing languages. When analyzing languages such as C# or C++, you find a lot of commonalities between the languages - frequently adding new syntactical features which converge the languages and generalize their uses. This comes at a huge complexity cost, and raises the difficulty for new programmers and experienced programmers maintaining complex software. Go was created to break away from the complexity of today's languages, not to overtake other languages and solve every problem, but to solve more specific problems such as threading, parallel computation, and networking. With a type system with no hierarchy, a structured package system with no conflicting feature sets, a software model with easy dependency analysis, and a clean and simple syntax, Go is a well designed language that shows promise.

Development Environment

One important problem Go solves is with its environment. Go is cross-platform and can compile for every (and from any) system and architecture. This means programming environments are irrelevant: Windows x64 can compile for Linux ARM. You don't need to worry about configuring build systems, setting up development environments, working against system restrictions or with system specific libraries, etc. As long as you have the compiler, you can compile with a simple "go install" command.

This is the same idea for Go's runtime environment. Go is a managed language with a full garbage collector, routine scheduler, etc; however, this managed runtime is statically built into executables in under 3MB. That means no redistributable packages or external runtime environments that need to be installed, such as in the case of QT, VC++, Python, or .NET. It compiles within seconds and runs natively for any targeted OS and architecture.

Chimera benefits from this light environment, as it can build libraries and executables for any system. It means the code is easier to setup and test, and easier to develop with across any system and with any Go developer. If Chimera was a company, this would mean less cost in training new developers, maintaining existing systems in Go, and less need for specialized systems for compiling code. If you're not familiar with large technology companies, these costs can easily be in the millions per year. Reducing these costs is of extreme benefit; but for Chimera, it's simply an added bonus for developers.

Development Reasons

Go specializes in parallel programming across networked systems. Chimera is a distributed system of servers which encapsulate the game world. Having a simple and managed infrastructure for connecting the map servers together was a large benefit, and meant saving months of development time in comparison to my C and C++ project variants. Not having to define a thread pool, database connection pool, etc. was extremely helpful. 

Go's package system also meant less time in development when writing ciphers and security routines, as Go comes with great cryptography and hashing support. In fact, the strength of their package system is that it is functionally complete - meaning no additional features will be added. This might be seen as a disadvantage, but my server design fits perfectly into their philosophy. With no additional features clouding development, there is always only one package for getting a task done in Go. This means that improvements to your application are normally delivered with the language rather than through refactoring. It means that you can be confident that your code is the best it can be for the language. Go also comes with a backwards compatibility guarantee - that your code will never break in language updates (or will be converted to work). 

Libraries in Go are also available through the open source community and delivered using Go's pre-defined interfaces. For example, to include a MySQL driver which defines syntax for Go's SQL connection scheduler, I include the official open source driver from github: "import github.com/go-sql-driver/mysql". This means Chimera no longer has to maintain external libraries which break across versions of VC++ and Visual Studio (for example with the C++ project). It just works and remains out-of-mind.

Technical Advantages

In addition to the advantages I listed above, Go is a very strong language for garbage collection and parallel computation. This helps eliminate my worries over multi-core execution. As mentioned before, Chimera doesn't focus on threading models or a thread pool any longer; rather, it depends on channels, Go routines, and Go's routine scheduler. Go routines are green threads which are scheduled by Go's statically compiled runtime. In recent tests, I was able to create a million of scheduled routines spin looping on a Raspberry Pi. It is incredible that the scheduler was able to not only avoid thrashing, but maintained a stable workload. Also, by relying on Go routines, I receive performance improvements with language updates.

I found in other tests that Chimera performed very well over the network. Pre-packaged functionality such as RPC and TCP delivered with the language were surprisingly well-written. When comparing networking routines with my C# variant, Go grossly out-performed C# but also was not susceptible to certain flooded attacks which I found crashed quite a few industry C# applications on local and remote systems.

Conclusion

By design, Go is more reliable, better on performance, and easily portable for a game server project such as Chimera. Go solves concurrent execution and communication specifications in Chimera with ease. It allows Chimera to scale quickly and keep a simple software model with less overhead and great performance. It also gives me the expansibility of a managed language with the feel of C. Though Go cannot be used to solve every problem, it is a great step in the right direction for network and parallel programming. And at the end of the day, it is my preference for maintaining a large distributed systems project such as Chimera.

Login Sequence Designed

Planning

Prior to moving forward from the account server, I decided to consolidate my thoughts on the login process by developing a flowchart. The flowchart below is what I'll be using next to implement the game and map servers. The primary purpose of this flowchart was to determine what exact roles the game server and map servers have in the login sequence. This helped me commit to the architecture I originally proposed.

RPC Servers

All servers will be communicating with one-another using RPC (remote procedure calls) over TCP. I completed the game server's RPC server just today, which allows the account server to send authentication data to the game server. This will be changed to the map server, given the new workflow I've defined below. Go's net package was great for quickly implementing this.

A question I receive surprisingly often is why I chose Go over other programming languages. I chose Go both knowing that my server architecture would benefit from the language and that I'd be more productive in the language. By using Go with Chimera, I'm able to eliminate a lot of the slowness and clumsiness I got from developing in languages such as C++ and C#. Go is my preference, and as long as I can remain productive in developing a scalable server using Go, I will continue using it for this project. Thanks.

Flowchart Diagram

Account Server Completed

Password Security

When observing private servers from this year, I noticed some still store passwords in plain text. I wanted to make it clear that Chimera doesn't store passwords at all (not as plaintext or ciphertext). Instead, Chimera securely hashes passwords using multiple key derivation functions upon registration, and uses that verifier to authenticate players. One key derivation function (KDF) in place salts the password using Go's cryptographically-safe random number generator and stores this unique salt in a separate database. These alterations make attacks such as rainbow tables and brute-force infeasible against the database. In addition, the server analyzes traffic and limits validation attempts from users exhibiting malicious behaviour. Chimera also employs SRP6 to securely authenticate users over the network without sending the password directly (rather using hashes).

Server Authentication

A major exploit found in most private servers relates to game server authentication. If done improperly, an adversary could then log into any player's character on the server. In the official server, players were authenticated by using an XOR of 0x4321 on the player's account ID, verified by performing the same XOR on the token to retrieve the account ID on the game server. This is not safe, nor is incrementing a number or encrypting the account ID like most servers currently do.

Chimera solves this security hole currently without any client modifications (this may change in the future). The transfer packet (msgconnectex) allows for a 64-bit token, which can be generated using a cryptographically-safe random number generator. I chose a cryptographically-safe random token rather than a one-time pad on the account id after a KDF since I'm attempting to protect against brute-force, and not a confidentiality breach. It is nearly impossible (or extraordinarily improbable) for this random token to be brute-forced in the amount of time required before the token expires (which is seconds, where a 64-bit integer takes roughly years to brute-force). On top of that, the server begins to reject authentication requests similarly to password verification.

And though this is more than enough to prevent brute-force attacks for such a short timeout period, I am still considering increasing this to a 128-bit token once the client hook is complete (which would take longer than the age of the universe to brute-force rather than years). It's not necessary at this time, so I'll weigh the benefits of implementing such authentication.

Account Access Log

Of course, the security I've mentioned above only functions if your password is secure, which a lot of the time isn't the case. For example, using the same password across multiple insecure websites and private servers. In the event that your account is "hacked" by another player who obtained your login from another server or site, Chimera performs extensive logging. These logs can be used for verifying such cases, but also help prevent them in the future as more security features are implemented before launch.

New Project: Chimera

Introduction

Chimera is a Conquer Online private server project written in Go. Roughly half a decade ago when I first began programming in a managed language, I grossly underestimated the requirements for developing such a project. I was unprepared for developing a server which could meet the performance requirements of my planned systems. Half a decade later, I look back at my old prototypes and miss the development scene. I have challenged myself again with the same goals for developing a high-performance Conquer Online server, though a lot as changed in regards to my design and choice in technology.

Architecture

The primary objective of Chimera is to create a distributed system of servers which service the Conquer Online game client. These servers are configurable to service a section of maps, and cluster together to encapsulate the game world. Below is a basic description of these server projects.

• Account Server: Authenticates connections and exposes a private interface for authentication in the game server project.
• Game Server: Routes players between regions and conducts synchronization with map servers; processes world events.
• Map Servers: Encapsulates a region of the game world and processes map and player actions.

Map servers cache changes to the world. When players exceed a calculated threshold for substantial character improvement, their changes are synchronized with the game server as a synchronization event. Other actions may also trigger a synchronization event. The goal in implementing this design is to minimize the overhead and requirement for larger synchronization periods and to minimize network stress and hits on the database. A side-effect of this architecture is that map servers may be migrated seamlessly to newer builds while continuing to serve players.

Packet Structuring

Another focus in Chimera is the scalability of internal systems, such as the server's packet structuring and processing procedures. To reduce development costs and time associated with defining procedures for reading and writing packets, I analyzed a sample of all known packet structures and constructed byte ordering rules for packet encoding. In combination with Go's powerful reflection features, these rules allow binary buffers to be generically converted to data structures, supporting recursive data types, collections, etc. Below is an example of this work in action.

Binary packet with header and payload:

00000000  1e 00 f7 03 28 13 00 00  15 03 02 47 6f 07 43 6f  |....(......Go.Co|

00000010  6e 71 75 65 72 07 59 6f  75 72 73 65 6c           |nquer.Chimera|

This packet can be represented by this simple structure in Go:
type MsgName struct { 
PacketHeader
Identity uint32
Action   byte
Strings  []string
}

And decoded using the following call:
err := packets.Read(buffer, packet)

Traditionally, I would be required to handle byte prefixed string lengths and reading in strings dynamically using custom byte ordering rules, but a system like this allows the me to simplify this process and improve the organization of the project.

Portability

Having a portable server architecture means easier deployment, reduced costs in maintaining system software / dependencies, and simplified requirements for selecting a host. Since Go is self-contained and cross platform, it can run on any operating system and architecture supported by Go; for example, across an array of ARM Raspberry Pis running Linux or Plan9. In addition to portability, Go offers Chimera a small and efficient runtime, compiled into the executable. The efficiency of this managed runtime and the Go garbage collector allows Chimera to be launched across low-resource systems, further adding to the flexibility of the distributed system.

Conclusion

I'll be updating this blog as I continue development. If you have questions, please don't hesitate to ask or make conversation; however, bashing on the Go language will not be welcomed criticism. There is no benefit to starting argumentation on language preference. All other feedback and criticism are welcome. To the question of "will this project be abandoned", there's always that possibility as I do attend school full-time and work part-time. Do I have intentions on abandoning it? No. My aim is to develop a server which is openly available. Thanks again for your interest and support.