Project Status Options

[Daniel]

dashboard v1.0 is currently live at http://dashboard.bastage.net

However, due to an ongoing issue with concurrent database queries failing due to transaction deadlocks, development of new features has been suspended pending the rewrite of the core engine.

Signatures and statistics collection from MTA will continue working. However, the live stats page may randomly disconnect you with a server busy message. Please do not raise this is a bug; it's a known issue.

Update 2010-07-17 02:15BST:
The new logging engine is running and collecting statistics and live status from MTA succesfully. The system can now tolerate a downtime period of 30 seconds (whilst the logging engine restarts) in the event of a failure of configuration change, and no data will be lost during these 30 seconds.

Addtionally, the new caching engine is up and running for data on the home page only. The home page of dashboard no longer communicates with a SQL database - it pulls data from an in memory cache instead.

More updates and improvements will be made this weekend.

Update 2010-07-19 01:16BST:
The logging engine has been further streamlined to support the concurrent decoding and logging of messages from MTA. The process is now split into four parts; the transmission of the message from MTA to the dashboard web application via HTTP, the transmission from the web application into the processing engine via WCF, the decoding of the message by a collection of IMtaMessageParser objects and then the updating of relevant caches and database by a collection of IDataOperationHandler objects.

It's all very long winded but effectively means we can send thousands of messages per minute into the engine and it'll process them in it's own time - there's no dependence on SQL Server, processing of messages won't affect the ability for dashboard to show data on the site, so everyone's a winner.

Tomorrow evening will be the transition of more MtaMessages from the old system into the new engine, and improvements to the caching engine.

Estimates are that we'll be able to handle around 10 million requests per hour on the new system.

Update 2010-07-23 00:12BST:
Migration of all the data collection functionality to the new pipeline has been completed. We're now processing all messages from MTA in a multi-threaded, asynchronous fashion meaning we tell MTA that message submission is completed once the message has entered the pipeline. We can then take as long as necessary to process the message and MTA is non the wiser.

The caching engine is also running and stable - all new messages are summarised and stored in cache, and also commited to database. When the engine starts, it retrieves raw stats from the database and performs calculations on these to populate the cache. The result is that after approximately 30 seconds after the engine is running, any data that dashboard needs to access is stored in RAM. This means we don't have to worry about transaction locks on the database.

More testing and implementation of some MTA "talkback" will take place this weekend.

Update 2010-07-24 03:02BST:
More late night coding! MTA now submits messages directly into the processing pipeline via WCF - it no longer needs to be able to talk to dashboard.bastage.net. The upshot of this is that the web interface now does *nothing* except show statistics to users; no processing, no relaying and no caching takes place in the UI layer any more.

I also implemented multiple server support this evening to permit RvB to log statistics as and when the time is right.

[warriorsuntd]

dashboard v1.0 is currently live at http://dashboard.bastage.net

However, due to an ongoing issue with concurrent database queries failing due to transaction deadlocks, development of new features has been suspended pending the rewrite of the core engine.

Signatures and statistics collection from MTA will continue working. However, the live stats page may randomly disconnect you with a server busy message. Please do not raise this is a bug; it's a known issue.

when do u think this issue will b fixed?

[Madis]

Probably just before he posts something about it here

[Arse]

OVER 9000!!!11!!111

nice work daniel

[Daniel]

Further update added.

[hobo pie]

heh "pipeline"

If livestats keeps being as sexy as it is, i may just have to lay 10" of my own "pipeline" somwhere

[Daniel]

Another happy happy fun time update!

[Rarekirby]

I cant log in to Dashboard.Why wont it let me log in?

[Daniel]

Go back to the top of the thread, read it, then answer your own question.

[wanty]

/me wonder's how it's it going the repair and when wil dashboard be up again.

[Daniel]

Not sure Wanty, we've got a lot of behind the scenes working going on at the moment. It'll be worth the wait though.

[wanty]

Not sure Wanty, we've got a lot of behind the scenes working going on at the moment. It'll be worth the wait though.

Nice, like to hear that.
Keep up the good job.

[=GLAYIS=]

dashboard v1.0 is currently live at http://dashboard.bastage.net

However, due to an ongoing issue with concurrent database queries failing due to transaction deadlocks, development of new features has been suspended pending the rewrite of the core engine.

Signatures and statistics collection from MTA will continue working. However, the live stats page may randomly disconnect you with a server busy message. Please do not raise this is a bug; it's a known issue.

Update 2010-07-17 02:15BST:
The new logging engine is running and collecting statistics and live status from MTA succesfully. The system can now tolerate a downtime period of 30 seconds (whilst the logging engine restarts) in the event of a failure of configuration change, and no data will be lost during these 30 seconds.

Addtionally, the new caching engine is up and running for data on the home page only. The home page of dashboard no longer communicates with a SQL database - it pulls data from an in memory cache instead.

More updates and improvements will be made this weekend.

Update 2010-07-19 01:16BST:
The logging engine has been further streamlined to support the concurrent decoding and logging of messages from MTA. The process is now split into four parts; the transmission of the message from MTA to the dashboard web application via HTTP, the transmission from the web application into the processing engine via WCF, the decoding of the message by a collection of IMtaMessageParser objects and then the updating of relevant caches and database by a collection of IDataOperationHandler objects.

It's all very long winded but effectively means we can send thousands of messages per minute into the engine and it'll process them in it's own time - there's no dependence on SQL Server, processing of messages won't affect the ability for dashboard to show data on the site, so everyone's a winner.

Tomorrow evening will be the transition of more MtaMessages from the old system into the new engine, and improvements to the caching engine.

Estimates are that we'll be able to handle around 10 million requests per hour on the new system.

Update 2010-07-23 00:12BST:
Migration of all the data collection functionality to the new pipeline has been completed. We're now processing all messages from MTA in a multi-threaded, asynchronous fashion meaning we tell MTA that message submission is completed once the message has entered the pipeline. We can then take as long as necessary to process the message and MTA is non the wiser.

The caching engine is also running and stable - all new messages are summarised and stored in cache, and also commited to database. When the engine starts, it retrieves raw stats from the database and performs calculations on these to populate the cache. The result is that after approximately 30 seconds after the engine is running, any data that dashboard needs to access is stored in RAM. This means we don't have to worry about transaction locks on the database.

More testing and implementation of some MTA "talkback" will take place this weekend.

Update 2010-07-24 03:02BST:
More late night coding! MTA now submits messages directly into the processing pipeline via WCF - it no longer needs to be able to talk to dashboard.bastage.net. The upshot of this is that the web interface now does *nothing* except show statistics to users; no processing, no relaying and no caching takes place in the UI layer any more.

I also implemented multiple server support this evening to permit RvB to log statistics as and when the time is right.

Hey daniel when i will see my stats:money,picture,kills,deaths,arrests,arested......?

[TheAmazonic]

GLAYIS:

1. Don't post that 2 times!
2. Don't quote that whole first post!
3. Answer to your question: When Dashboard is online again.

[Daniel]

Hey daniel when i will see my stats:money,picture,kills,deaths,arrests,arested......?

3. Answer to your question: When Dashboard is online again.

Not quite. If you've populated the "in-game username" field on your forum profile, click through to your player stats page (blue text next to each of your posts) and your signature image is there. Copy the address and stick it in your signature.