Results 1 to 10 of 10
  1. #1
    Join Date
    Jan 2010
    Location
    Buenos Aires, Argentina
    Posts
    148

    Performance Concern on MMO asset DB system

    I've just finished watching the videos about the persistence system.

    While Nelson is doing a great job (as always) I'm a little concerned about the performance of these system.
    Maybe I'm jumping the gun on this, since there seams to be some advanced features not yet covered on the videos.

    My concern is basically I/O (server side). With potentially 1000s of users pulling assets from the db, IOPS are going to be very high( very very very high) and if the DB gets fragmented the seek times are going to be a problem too.

    I have some idea on how to tackle this issues, like multiserver zone sharding, readonly memory copies, readonly SSD arrays (to avoid seeking), Master/Readonly Slaves clusters, etc.

    I just wanted to ask if you already thought about these potential issues or if these is not important right now since these is in R&D stage?

    PS: I had to switch at least 2 gears when I started watching Nelson's session 11... Gosh, Nelson codes fast!

  2. #2
    Join Date
    Mar 2007
    Location
    Las Cruces, New Mexico
    Posts
    43
    At first I thought it was funny how fast Nelson types, but it got old pretty quick. He also tends to scroll several times as soon as he finishes so it's very difficult for me to see what he just did. He also tends to talk very implicitly, so I am forced to watch what he is typing and am left behind whenever I look at my own screen. He may be a great developer and able to win a typing contest, but he has a lot to learn about presenting information.

  3. #3
    Join Date
    Jan 2010
    Location
    Buenos Aires, Argentina
    Posts
    148
    Quote Originally Posted by eibonscroll View Post
    At first I thought it was funny how fast Nelson types, but it got old pretty quick. He also tends to scroll several times as soon as he finishes so it's very difficult for me to see what he just did. He also tends to talk very implicitly, so I am forced to watch what he is typing and am left behind whenever I look at my own screen. He may be a great developer and able to win a typing contest, but he has a lot to learn about presenting information.
    Give Him some time. He is getting better at presenting the content. It's not so easy to explain the complexity of what's Nelson building. But he is trying hard to explain how he codes and at the same time giving some cool Resharper tricks. I see a big improvement in Nelson's VTMs from the appdev videos to this ones.
    And you have the time slider to catch something you've missed.

  4. #4
    Join Date
    Mar 2004
    Location
    Anacortes, WA
    Posts
    4,168
    @OP

    Just to re-iterate what we talked about in the IRC, we will soon go into more depth as far as how this system is going to be used. But your concerns are of course valid.

    Quote Originally Posted by eibonscroll View Post
    At first I thought it was funny how fast Nelson types, but it got old pretty quick. He also tends to scroll several times as soon as he finishes so it's very difficult for me to see what he just did. He also tends to talk very implicitly, so I am forced to watch what he is typing and am left behind whenever I look at my own screen. He may be a great developer and able to win a typing contest, but he has a lot to learn about presenting information.
    You are somewhat correct.

    I want to explain things from my perspective. When I code, I type at 90+ WPM - and the shortcuts and tools that I use make that even faster. Learning to code differently isn't easy for me, but I realize that as far as instruction goes, it would be better for most people if I slowed down.

    That being said, there are many, many types of developers that I must cater to in these videos. I do a lot of one-on-one with people - and the way I speak or type will be different depending on who I am explaining things to. For people that I know personally, it is easy for me to change the way that I talk and type so that things work better for them - but when I have to explain things to an undefinable variety of people, things get more complex.

    The audience of Nelson's Corner, for example, was fairly well defined. I almost never got any complaints from people, as the people who watched those videos knew exactly what they were getting themselves into. I could make assumptions about people's knowledge and comfort of code. The MMO... that's a bit different. We have developers who just started programming, and developers who have been programming for years. Finding a balance between these groups is hard. Within those groups, we have developers who never ventured into more advanced language concepts as they felt them unnecessary and esoteric - yet we also have developers who's first instinct, upon receiving a managed assembly, is to scrutinize the IL itself. We have performance experts, architecture experts and everything in between. We have hobbyists who have never developed under a deadline, and professionals who put a dollar value on each line typed.

    Do I simply go over the code, show people the external API and glance over the implementation? This is how I would, personally, like to be taught to. For me, the code itself isn't going to be an issue - and many things can be left unsaid. The things that do need explaining need not go into hour long discussions about how different features of C#/.net were used, and if they did, it would only frustrate the viewer (myself).

    Do I avoid generics, FP and OOD? People who know how to program, yet haven't gotten into more advanced features, would love if I did - as they could now understand the concepts and algorithms without having the extra overhead and mental tax of having to read code that they are uncomfortable with. In this case, the speed in which I type wouldn't be the issue - as these developers should already know enough logic to get something out of the code on the screen without being burdened with new concepts left and right.

    Do I avoid tooling altogether? Non-.net devs and people new to R# or VS would love if I did. Watching code appear without understanding the context of how it got there would require much more backtracking on their end - and minimizing the use of shortcuts and IDE features would be much less confusing on their end.

    Do I use tooling, but explain every shortcut, every menu, every... everything about it? I think a lot of people would appreciate this, but what would suffer is the length and focus of the videos.

    What assumptions can I make about a group so diverse? What tools and features can I use to produce production-quality code in a short timeframe (as remember, the MMO is going to be a product, by 3dbuzz)? How much time do I spend explaining things - and what things should I spend that time on?

    What has happened, and what you have seen, is me switching between these styles - often times, within the videos themselves. I realize that this helps nobody - and I am going to settle down on a style eventually that caters to the widest audience. Jason has already given me specific tips as to what he would like to see, and people have responded positively to these changes. But there is more that I can do, and I realize that.

    I just want you to realize that this isn't like talking one-on-one to a single developer. This isn't like doing a company-wide presentation, when I know the average strengths and weaknesses of the group. You can't raise your hand and ask me to skip something that you feel I'm wasting time on - or ask me to go over something that you feel I skipped over.

    I have spent a lot of time thinking about the best way to present information in this setting. What I think you don't understand is the challenge in doing so to such a diverse group.

    That being said, I will listen to any critiques that you have and I would appreciate any time that you could offer in telling me what specific things that I could improve on, in your eyes.
    Need any help? Feel free to PM me - or send an email directly to nelson@3dbuzz.com!

  5. #5
    Join Date
    Aug 2011
    Location
    Kansas
    Posts
    181
    While I agree with the others who said you covered things pretty quickly, after a couple of viewings of the Chapter 1 RavenDB videos, I caught on to what you were doing and was able to figure it out. (I don't have any other chapters unlocked yet)

    Back to the original poster's concern - how well does RavenDB scale up for the increased demand of an MMO? Have you tested it under high-demand conditions at all? Or is that coming later?

  6. #6
    Join Date
    Mar 2007
    Location
    Las Cruces, New Mexico
    Posts
    43
    @Nelson:

    Thanks for making some changes. It's much appreciated.

    I do understand the difficulty. I'm a professional developer and manage other software engineer's. I find myself in front of a white board teaching a concept quite often. It's also the case that one of the engineers listening, while not having specific knowldege in what I'm presenting, say some specific game AI algorithm, may have a broad and very deep understanding of Discreet Mathmatics and Graph Theory. It's hard to say enough, but not say too much.

    I completed number 6 of the World Data Storage. And pm'd Zaq a positive response to the change I saw in your presentation from the RavenDb video set, which is the one I and a co-worker had a very difficult time with, mostly due to the code not being on screen long enough to take a look at. I was going to wait to append to this thread until I got completely through the new video set.

    I did make some adaptions that helped me and recommend to others to use the right tools if they are having problems as well. I switched from VS Express to Pro and installed Resharper. I'll probably do your video on Resharper to use it more productively.

    The biggest issue that I had was that I couldn't see the code on the screen, in RavenDb video's it was hard to even get a pause in the right place. At the end of WorldDataStorage 4 you left the classes you coded up at the end and I thought that was awesome, however, because of the lack of fast scrolling and my switch to using Resharper I was able to pretty much keep up during the video and didn't require it. That's one feature that I think it would be great if you kept.

    I do recognize the difficulty in what you are doing, at least part, and see you trying out different things in your latest MMO video set that made a huge difference in there watchability. I think excessive scrolling after a section was coded quickly is what I had the hardest time with honestly. Just hanging for a few seconds before you scroll makes a huge difference. Not knowing what is in your head, ahead of time makes multiple rewinds necessary to do what should be a trivial task such as creating a directory under directory x while setting up a project. I think some necessary tedium is needed in doing video's like this, it's just the nature of it. Going a little slower, actually can save more time for the viewer.

    One thing I normally found frustrating in CS classes is when the teacher tried to program in front of the class. Even the most brilliant of teachers would end up having to undo things, backtrack, and rewrite. After 3 or 4 iterations of a function most people had lost what had happened and an explanation of "there you go" never quite did it. Now, one of the stated goals of the MMO class is to show "over-the-shoulder" development but I don't think that's possible in reality. I know Lee prepares his videos off of code he has allready written and has available on another screen.

    The very best example of teaching programing to a diverse group that I have seen is Mehran Sahami, of a cs 101 course. I'm not saying to copy him, but he's a really good cs lecturer. Here's the link http://see.stanford.edu/see/lecturel...a-866adcae1111

    Not to high-jack this thread.

    I think I am going either with Oracle or MySQLfor the project I am working on. It's not based on performance tests but just that there are too many unknowns for me with RavenDb. Hiring someone with experience with it is also a concern.
    Last edited by fatgav; 08-30-2011 at 02:28 AM. Reason: double post

  7. #7
    Join Date
    Jan 2010
    Location
    Buenos Aires, Argentina
    Posts
    148
    I wasn't talking about ravendb. I was talking about the terrain persistence system developed by Nelson on chapter 6. RavenDB makes a lot of sense for player/game data.

  8. #8
    Join Date
    Mar 2007
    Location
    Las Cruces, New Mexico
    Posts
    43
    Sorry, I missunderstood you. I jumped on your P.S. about having to switch gears and didn't read your original question close enough.

    I have not yet seen a Terrain System that performs well for large terrain sets. I'm only through #6 of this series so can't speak to it.

  9. #9
    Join Date
    Oct 2006
    Posts
    45
    Nelson is awesome. He doesn't screw around - its straight-up brain to code that assumes you've got basic knowledge of patterns and C# itself (which is a reasonable requirement for this course).

    Also I agree that the terrain store is utterly insane. I'm doing a similar thing, but with appropriate spatial partitioning, a base heightmap and a delta heightmap in a grid-nested quadtree. It's fast as hell and doesn't really concern itself with pixels - it only cares about the regions they represent. The base map is half the resolution of the delta map, but could be the same or twice or whatever.

  10. #10
    Join Date
    Oct 2006
    Posts
    45
    Quote Originally Posted by eibonscroll View Post
    The very best example of teaching programing to a diverse group that I have seen is Mehran Sahami, of a cs 101 course. I'm not saying to copy him, but he's a really good cs lecturer. Here's the link http://see.stanford.edu/see/lecturel...a-866adcae1111
    All of that Stanford stuff (which is also on itunes as free pod/vodcasts) is a gold mine.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •