Nightgames Mod (v2.5.1.2) updated 2/11/17

[ThatOneJester]

So I was ambushed by Airi, and the character image was Rosea's.


Is this being used as a placeholder, or is it a bug?


Edit: Right so I continued playing to level up my character some more, and when I fought Rosea the character portrait was Jewels.


I think this is a bug where the portrait changes to the last person you fought.


Edit 2: It seems I have so many traits that my statuses get cut off, will you be adding a scroll bar to this as well? xD

 

[dndw]

Okay, so as may have been surmised, I'm working on getting this game to Android. There's a small problem, however. I have a reasonably powerful phone (or at least it was when I bought it), but simulating an AI match takes upwards of ten seconds. Ten seconds in which the game simply freezes, at least a dozen times a night. That won't do. I've already halved the maximum amount of turns to simulate, but it's still too slow. Now, I could cut further into this, sacrificing accuracy for speed, or I could do away with simulated matches entirely and think of some mathematical approximation instead. That way, there would just be a semi-random formula into which the requisite variables get plugged, and out pops the winner. Both options will take time to implement. So my question is, would you guys mind waiting a few seconds for proper simulations, or would it be better to 'cut corners' like this so you don't have to wait?

 

[ThatOneJester]

Cutting corners always forms problems later on, so I would say to just take your time with it.


It isn't like your games fans are a bunch of impatient 10yr olds.

 

[dndw]

Oh, I wasn't talking about development time; that's going to take a while either way. I was talking about waiting while playing: phones aren't too powerful and simulating the matches takes them a while. When I said 'cutting corners', I meant doing away with the simulation and doing a much more simple (and less realistic) calculation instead. (And it isn't my game, by the way)

 

[invalidcharacter]

Would making the offscreen battle's RNG fixed to an initial randomized value make the matches significantly faster? (have yet to check how it works exactly)

 

[The Silver Bard]

Properly simulating an offscreen battle is a luxury that modern PCs allow, but isn't that important to the game design. I'd recommend replacing the NPC-only battles with a simple weighted die roll. Also consume a few random items from each participant to retain item balance.

 

[dndw]

Properly simulating an offscreen battle is a luxury that modern PCs allow, but isn't that important to the game design. I'd recommend replacing the NPC-only battles with a simple weighted die roll. Also consume a few random items from each participant to retain item balance.

Yeah, I figured that would probably be best. It'll be a challenge to incorporate every aspect of a character into such a roll, but it should be possible.


In that video I posted, an observant viewer might have spotted my email address in the autocorrect. That address isn't secret or anything, which is why I didn't blur it out. Turns out, however, that shortly after I posted it someone living in or near Boardman, Oregon, United States tried no less than 12 32 (!) times to access my account. I figure there's a pretty good chance whoever did so will read this as well, so I wanted to say that it is not appreciated. To show you just how much it isn't appreciated, I shall leave your IP address here:  52.33.135.19. Microsoft apparently likes to store these.


Edit:


This is my preliminary idea for insta-resolution:


Every skill supplies a score of 0 to 10 for each of four 'resources': Stripping Capability, Debuffing, Mojo Generation and Positioning.


The average of these scores for all available skills is calculated, and in turn used to calculate a 'skill quality'. This calculation involves looking at what is needed for a skill to succeed, versus how good a character is at supplying it. In a sense, this is a measure of how likely the skill is to be usable during a match. This score is compounded with the usual priority modifier, to get a sense of how much benefit the skill will have. All such qualities are then combined with a score based on the condition of the character (stamina, arousal, mojo, willpower). Finally, a random bonus is added to the scores, where higher levels are more likely to get higher bonuses. If the final scores are very close, it's a tie. Otherwise, the highest score wins.


I still have to fit clothing, traits, statuses at the start of the match and body modifications into this


Any suggestions? Ideas?

 

[xayaz]

Before we go making bigarchitecture changes, let's see if we can just improve the performance of the existing combat implementation. The combat calculations aren't complicated; they should be able to go pretty quick.


I haven't worked in Java for a while, so I'll need some time before I can make focused contributions. My initial performance testing with the Java VisualVM (http://docs.oracle.com/javase/8/docs/technotes/guides/visualvm/intro.html) indicates that a lot of time is being spent in Character.clone() and Body.clone(). I'll have to look into whether the existing clone implementations can be improved, or the number of clones can be reduced.

 

[The Silver Bard]

Oh that's right. The mod's AI uses that optimization algorithm. That has to be really resource intensive. You may want to start by disabling that in NPC-only matches and see how that performs.

 

[dndw]

Oh that's right. The mod's AI uses that optimization algorithm. That has to be really resource intensive. You may want to start by disabling that in NPC-only matches and see how that performs.

Ah, sure, leave it to me to over-complicate things. It still takes a few seconds in 'dumb mode' (I did not create that term, I suspect Jos), but that's acceptable. That's also with a limit of 25 turns, but total realism is not too important, I guess.


Now let me go fix the UI that I've just inexplicably and totally destroyed without so much as looking at the code.

 

[invalidcharacter]

Shit, wait, I was working on the creation file right now. Are you working on it as well?


(CreationGUI)

 

[dndw]

Shit, wait, I was working on the creation file right now. Are you working on it as well?


(CreationGUI)

Nope, I'm working on the Android version, which is an entirely separate project. There is no such thing as CreationGUI.

 

[invalidcharacter]

Ah alright. I'm hoping to finish this up today, before I have to delay it further.

 

[invalidcharacter]

Preview. This is by no means the final look it will have (just what I could get done today), but things are at least in the general location they should be at (had a bunch of cleaning up to do). On the upside the current model is a godsend for low resolutions, on the downside it's because literally most of the window is a bunch of giant buttons. I am *really* tempted to let them stay fairly large though, because for reasons I like to play this on a surface pro 4 with a touchscreen.


If you have feedback, I'd appreciate it now. At this stage I can change pretty much the entire thing with ease. The intro text will show up after this screen.

 

[xayaz]

Oh that's right. The mod's AI uses that optimization algorithm. That has to be really resource intensive. You may want to start by disabling that in NPC-only matches and see how that performs.

Yeah, it sure looks like prioritizeNew represents a large percentage of runtime. It allocates a new combat clone, and therefore two new character clones, five times for every available skill, every round. It'd be a nice algorithm in a pure functional language, if the game were engineered quite differently. But as it is, it's extremely expensive, even on desktop. I'm looking at improvements; should be able to shave 50% or so off the time.


I did catch and fix a separate CPU hog: PR #43

 

[xayaz]

Could someone post a late-game save from 1.9.x? In the early game, the StripSelf skill takes about 20% of the combat simulation time. (rateAction() again.) I'd like to get an idea if that remains the case later on, and if there are other skills that could be improved.

 

[dndw]

Preview. This is by no means the final look it will have (just what I could get done today), but things are at least in the general location they should be at (had a bunch of cleaning up to do). On the upside the current model is a godsend for low resolutions, on the downside it's because literally most of the window is a bunch of giant buttons. I am *really* tempted to let them stay fairly large though, because for reasons I like to play this on a surface pro 4 with a touchscreen.


If you have feedback, I'd appreciate it now. At this stage I can change pretty much the entire thing with ease. The intro text will show up after this screen.


View attachment 2800

I don't really like it, honestly. Primarily because of the huge buttons, as you suspected. Perhaps if you ask The Silver Bard really nicely, he'll let you use the new title image.

Could someone post a late-game save from 1.9.x? In the early game, the StripSelf skill takes about 20% of the combat simulation time. (rateAction() again.) I'd like to get an idea if that remains the case later on, and if there are other skills that could be improved.

I guess a reasonable strategy would be not to do these extra simulations, but rather to make a guesstimate of how good every subchoice would be. Another one could be to drop the skills with low priority mods before ever resolving them.

 

[invalidcharacter]

I don't really like it, honestly. Primarily because of the huge buttons, as you suspected. Perhaps if you ask The Silver Bard really nicely, he'll let you use the new title image.

They do look extra silly. I'll finish up the formatting, post a reasonably sized/formatted version and if you still don't like it pursue a different look entirely. Might keep this one for my sp4 though.

 

[invalidcharacter]

Couple of tweaks later, looking a lot more like the final result I had originally intended (still missing all the polish). Blame the absurd lack of DPI scaling support for it looking blurry, I made it so the display is 720p in the screenshot.


I'm really, really tempted to manually change every last font size to it starts resizing it over 1080p. I can't read anything on this monitor.

 

[xayaz]

Actually, the clone() performance problems weren't intractable. I posted a tracking issue and pull request to Github, reducing the clone time to about a third of what it was. Basically, I moved the cost of cloning the HashSets from clone() to the points where those sets are actually modified (which is much less often). Functional programming FTW.


I've got more optimizations in mind for other parts of the sim. I should be able to improve the performance by another 20-40% across the board.

 

[nergantre]

Actually, the clone() performance problems weren't intractable. I posted a tracking issue and pull request to Github, reducing the clone time to about a third of what it was. Basically, I moved the cost of cloning the HashSets from clone() to the points where those sets are actually modified (which is much less often). Functional programming FTW.


I've got more optimizations in mind for other parts of the sim. I should be able to improve the performance by another 20-40% across the board.

Nice.


There are a number of improvements that could be done for the AI sim actually. For one thing, we shouldn't even need to clone the characters. Implementing a character.copyof or something would be probably good enough, where you just keep two characters. There may be some pruning you can do for the skills to simulate as well... Although, since it's currently at depth = 1, I'm not sure how much that matters.


I haven't really touched performance because it hasn't really been much of a bottle neck, except for deep decision searching, which honestly would take more work to implement in a nice way anyways

 

[Iloei]

I sent a pull request (#42) with some textual changes, and I do not know if that was good, bad, or should have been done differently. Can someone give me a hint?

 

[xayaz]

For one thing, we shouldn't even need to clone the characters. Implementing a character.copyof or something would be probably good enough, where you just keep two characters. There may be some pruning you can do for the skills to simulate as well... Although, since it's currently at depth = 1, I'm not sure how much that matters.

The underlying problem is that the combat actions modify the characters in-place. We don't have any way to undo those changes and get back to a clean slate, so we have to make a new deep copy for every simulation, otherwise the changes would stick around and the simulation result would be messed up.

I haven't really touched performance because it hasn't really been much of a bottle neck, except for deep decision searching, which honestly would take more work to implement in a nice way anyways

Right, the decision searching is the only performance problem.


Don't get me wrong: I think prioritizeNew() is implemented in a nice way.  For a collaborative mod project, I don't think there's really any other way to get the AI to sensibly use the wide variety of moves available, in the various situations they find themselves in. It just requires that rateMove() and everything under it be fast.


This is the correct way to make a project: write it so it works, and then go back with a profiler, see what's slow, and fix that. I'll spend a few more days doing the latter. It's fun.


(For another example,  BasePersonality.getType() is now a pretty significant hotspot, taking 8.5% of the total CPU time over the course of a night after my other performance fixes. Turns out that getSimpleName() is actually an expensive operation; whoda thunk. But it's easy to cache the result in a field and completely kill that particular problem.)

 

[nergantre]

The underlying problem is that the combat actions modify the characters in-place. We don't have any way to undo those changes and get back to a clean slate, so we have to make a new deep copy for every simulation, otherwise the changes would stick around and the simulation result would be messed up.

Yup, but there is still not really a need to alloc the entire character per move. Implementing something like copyof and using a single character object for the battle sim and resetting it after each move should be sufficient. 


That's assuming that the jvm even does allocs on new though... I might just be bullshitting here haha.


Thanks for doing this by the way, every time I open up eclipse I think I should probably do a performance pass, but I end up just doing something else anyways

 

[xayaz]

Implementing something like copyof and using a single character object for the battle sim and resetting it after each move should be sufficient. 


That's assuming that the jvm even does allocs on new though... I might just be bullshitting here haha.

Oh, I see what you mean. I had thought about that, but I don't think it'd be worth the effort. The jvm does alloc on new, I believe, but it's got a pretty good garbage collector so it's not a big deal. It's probably a bigger problem on Android, though. But just reusing the Character object wouldn't help much; most of the time is involved in cloning and rebuilding the various HashSets, HashMaps, etc. Which is sometimes unavoidable because the items are mutable, or refer to the other Character in the combat. This ends up being expensive because it's got to call toString() every time to calculate the hash.


On a related note: thoughts on importing Google's Guava library into the project? It's got some nice collection types and algorithms that would make it a bit easier to refactor things into an immutable style. May even end up making things faster. For example, I just rewrote Body.get() like this:


public List<BodyPart> get(String type) {
return Lists.newArrayList(Iterables.filter(currentParts, p -> p.isType(type)));
}


Lists.newArrayList and Iterables.filter are both from Guava, and currentParts is an ImmutableSet that I update from bodyParts and the replacements whenever anything changes. This is a hell of a lot faster, but to be fair a non-Guava solution probably wouldn't be that much slower. Just more confusing boilerplate. (See this StackOverflow question for the equivalent non-Guava code.)


(If I were really ambitious, I'd change it to just return the Iterable rather than an ArrayList, but that'd require refactoring a bunch of call sites, and it's not a hot spot any more.)


The downside would be that it'd add two or three megs to the source and distribution. (There's also a bunch of other good stuff in there.) I'd do it if it were my own project, but I don't want to push anything on the rest of the community.

 

[dndw]

On a related note: thoughts on importing Google's Guava library into the project? It's got some nice collection types and algorithms that would make it a bit easier to refactor things into an immutable style. May even end up making things faster. For example, I just rewrote Body.get() like this:



public List<BodyPart> get(String type) {
return Lists.newArrayList(Iterables.filter(currentParts, p -> p.isType(type)));
}


Lists.newArrayList and Iterables.filter are both from Guava, and currentParts is an ImmutableSet that I update from bodyParts and the replacements whenever anything changes. This is a hell of a lot faster, but to be fair a non-Guava solution probably wouldn't be that much slower. Just more confusing boilerplate. (See this StackOverflow question for the equivalent non-Guava code.)


(If I were really ambitious, I'd change it to just return the Iterable rather than an ArrayList, but that'd require refactoring a bunch of call sites, and it's not a hot spot any more.)


The downside would be that it'd add two or three megs to the source and distribution. (There's also a bunch of other good stuff in there.) I'd do it if it were my own project, but I don't want to push anything on the rest of the community.

I haven't used Guava much before, but the above can be accomplished in 'regular' Java just as easily:


public List<BodyPart> get(String type) {
return currentParts.stream().filter(p -> p.isType(type)).collect(Collectors.toList());
}


For immutable collections, you could use (*deep breath*) CopyOnWriteArraySet or the equivalent List. I don't know which option is faster.


For the cloning, you could instead create secondary lists. I envision the following:


    - Create a 'CharacterClone' class, which stores initially empty collections of traits, clothing, etc. and give each character an instance.


    - Whenever a mutation occurs to one of these collections in a cloned character (as denoted by the 'cloned' field), update the CharacterClone instead.


    - For every read operation on a clone, check both the original and CharacterClone collections.


    - When re-cloning, just empty all the data in CharacterClone


This might not actually be cloning anymore, just a lock of sorts. We'd need to be extra careful to make sure everything gets passed to the CharacterClone, though. One slip-up will break stuff. That's the same as it is now, except there no longer are any slip-ups. Remember the time you'd randomly grow body parts during a match? This was the cause.

 

[Pim_gd]

Streams are Java 8, which doesnt work on android yet as far as I know. So keep that in mind.

 

[dndw]

Streams are Java 8, which doesnt work on android yet as far as I know. So keep that in mind.

Oh, I've noticed. I'm proposing this for the regular version, and I've already had to go through all of the code to replace the Stream API calls with pre-Java 8 equivalents. Lambdas can work, fortunately, with a neat plugin which converts them pre-compilation.

 

[dndw]

So, first test for the Android version. You can find the apk attached to this post.


To install, you'll need to either download it from your phone, or use a cable or something to transfer it from your computer. You'll also need to allow your phone to install non-market apps (Settings > Security > Unknown Sources).


It should work from Android 2.3 up, but no guarantees beneath 4.0.


When you start the game, press 'Play Mod', as the original isn't in yet. If you want to load a save, you'll have to place it in the <internal memory>/Night Games Saves/ folder. This folder will be created when the app first starts. You'll have to remember the name of the file, as Android does not have a file picker. (Seriously Google?)


In-game, everything works mostly similar to the game we all know and love. To see your attributes, traits, etc., swipe right-to-left. To see the options, load button, etc., swipe left-to-right.


Swiping back from the option screen to the main one is a bit tricky, you'll have to find some open space. I recommend the bit between the bars and the buttons/text.


When you're done, I'd really, really appreciate it if you could fill out this quick form. It's mostly to see what screen types and Android versions are problematic. I've tested it on a 4.7'' 1280x768 screen with Android 5.1.1, and it looks like other screen types should work as well, but emulators are far too slow for extensive testing.


Edit: I think the mod version used is 1.9.3, but I had to edit quite a lot so I'm not sure


View attachment Night_Games.apk

 

[xayaz]

CopyOnWriteArraySet doesn't actually implement Cloneable. There doesn't seem to be any way to make a shallow copy. CopyOnWriteArrayList does support clone(), so I used it and replaced all calls to .add() with .addIfAbsent(), which isn't documented but would seem to give the same result. Seems to work fine in testing; any idea if this is unsound? Thanks for the point about .stream(); your implementation works great.


I can post a PR if anybody's ready to review and merge. There actually aren't many changes; no Guava, mainly replacing a few fields with CopyOnWrite versions, caching a few things, and some clean-up. Let me know.


It's hard to benchmark consistently, but I'm seeing combat times fall by 70%-80%, which is a pretty noticeable improvement when you're moving around the map. No more five-second hiccups.


The remaining performance fixes are difficult. Character.effects is a sticking point. I read your proposal, dndw, but (if I understand correctly) the big problem is that each Effect is itself mutable (and carries circular references to its Character and the other Character, to boot, both of which it can mutate). Just monitoring the collection for changes wouldn't cut it; we'd have to intercept and somehow rewrite changes to the effects themselves. There may be a way to do it---remember, I'm not familiar with Java---but it looks like a mess to me. I'd "just" rewrite Effect to be immutable, but that'd require reworking half the game.


Oh, I also removed all the unused imports across the project. The remaining six warnings all look like genuine bugs.