Larian Studios Forums Divinity - Original Sin Divinity Original Sin - Mods Trouble with dialog system

Using a smallint to store IDs in a game with millions of objects ? Yuck ... ;-)

The problem I had was the notorious 'game crashes on zoning' thing, not everyone had that, but I'm a collector and collect everything that can be picked up. (And I also convert stackable iron bars to non-stackable weapons ;-)
DOS is an amazingly stable beast normally (and very resistent to my modding errors too ;-) and when it happened on my first playthrough I thought 'Crashed ? Can happen! Maybe the save has been corrupted', but when it happened in the next playthrough too, I knew something was wrong.
Anyway, my solution back then was to destroy most of the collected stuff with crafting, or convert it to something else via crafting so that it would stack.

When I examined root templates after the patch that supposedly fixed the problem last year, I noticed that they had made a lot of stuff stackable that was not stackable before and increased stack sizes for other stuff. So my assumption was, that there were simply too many different items and some overflow in trader's inventories caused the crashes. Since it only happened on zoning, one time locking out Cyseal, the other time Luculla for me, I can only assume that the stuff I carried around in inventory added to the 'items per zone' which caused the crash. And I am sure I was very far away from 350 hours, even though sometimes I just had the game running in background or in foreground while taking a smoking break ;-) (I had even chosen Cecil's town hall guards and normal civilians to sell the 1 gold junk to in order to prevent traders from crashing, did not help my own 'fixes' ;-) (As far as I remember my first successful playthrough was even before the patch, with my own changes to stack sizes and item destruction in crafting.)

Anyway, since it is so easy to delete handles, I just do it. If done right, can probably do no harm.
('Black hole trading' makes trader's inventories less of a mess if you ever want to buy something, so why not also delete the stuff. Important stuff is not sellable anyway.)

Since the 'handles' I can access in Osiris are no real 'item handles' but more so 'handles of container slots', maybe it's the number of used inventory slots. (Slots in containers and character inventories.)
It would definitely make our lives easier if Larian ever gave us infos ;-)

One patch made a change to which items were global, which made the issue of running out of handles easier to hit with a lot of trading or crafting. A subsequent patch increased the limit to 64k, but it is still possible to reach that (a couple with playtimes on the order of 350 hours, and one person trying to make a mod to have traders regenerate their inventory each time a character talked to them).
One person ran into this problem in D:OS Classic, as well, though only after it was fixed in the EE, with a playtime about 550 hours.

Thanks for stepping in with some info and my big respect for helping us in your free time. (on a weekend, 5 in the morning in Belgium!)

I'd have a couple of questions on the subject ... for after your weekend ;-)
- do all placed objects in the world have a handle too ? (Like trees, bushes, any stuff that cannot be picked up. They could still be uniquely identified by their UUID without a handle and some scripts do that. If they used one, that would greatly reduce the pool of handles for all other processes.)
- does ItemHandleDelete() free a handle ?
- to which dump do items go that were removed with any of the ItemTemplateRemoveFrom... calls and are their handles freed ?
- can freed handles be re-used by the game or is it a simple 'count up without looking back' thing ? (like the sequences relational database systems often use to create PRIMARY KEYS) If they could be freed and reused, a trader who actually cleans up would help a lot, converting many items to a few ones that stack high with crafting would help as well.
- 'handles' in Osiris are no single item handles but handles to an object that can occupy one slot in any container (including a character inventory) and that's the reason why they have to be moved away with some ItemHandleMoveTo... call before they can be treated as single items ?
- do global items and local ones have two different pools of handles and if so, could the global pool even ever change when the game is running ?
- can the code be adjusted to use int for handles instead of unsigned small ? The limit would be a tiny bit harder to reach ... (okay, just kidding ;-)


Anyway, enjoy wour weekend

I'm in Canada, not Belgium (though it is after 5am here now; I got up and had to check the forum/email on my way back to bed... ). I work evenings and weekends, since that is when more people play games (and working late I can overlap with the morning in Belgium).

Not all items have a handle, only global items. Picking a local item up and dropping it will create a handle for it.

Deleting a handle does not allow it to be reused.

I don't know about the removed items. I can ask about that, if it matters in light of the previous answer.

The handle generator uses two parts, one of which is a global counter (which only increments each time it is called), and doesn't keep track of or look for handles that are no longer used.

I don't know about ItemHandleMoveTo.

AFAIK local items don't use a handle.

I was told this could not be fixed for D:OS, but that would be changed for D:OS 2. I don't know if that was purely for technical reasons, or the chance of major changes breaking things for others was too great a risk given the relative rarity of issue with the higher limit.

Canada, interesting. Did Larian specifically pick someone living overseas for the job so that 'normal' working hours would overlap with european playing prime times ?

Thank you anyway for answering questions on a weekend !

So what you say means that the engine never does some kind of 'handle space defragmentation' which inevitably leads to death at a certain point, although there might still be a lot of handles potentially usable, but the engine never bothers to look for the 'holes'.

A handle needs to be created at the latest when the item is put into some container, independent of the item being global or local (globals might have a handle created on game start to be fully prepared when Osiris uses them). Otherwise we would not be able to get the handle in Osiris 'events' like ItemTemplateAddedToCharacter( (STRING)_ItemTemplate, (INTEGER)_ItemHandle, (CHARACTER)_Character ).
All these 'events' are queued as soon as an item is put to any container.
Which makes me think, that it's actually a slot that can hold an entity (a single item or a stack of items) which aquires a handle, attaches it to the entity and sends info about all this to Osiris. It also sends who or what its parent is, so Osiris can categorize it as either an ItemTemplateAddedToCharacter() or ItemTemplateAddedToContainer() 'event'.

I was also wondering if the change to treasure creation from classic to EE is at least in parts responsible for the overflow crashes (which I had never heard of before in classic), because with the early creation on zoning instead of 'on-demand-creation', a lot of handles might become unnecessarily wasted, which could lead to the limit being reached much earlier than before in classic.
The fact that people (me included) had crashes on zoning also suggests that the new treasure creation is the cause for the overflow.
Of course without ever 'defragmenting the handle pool', this problem could have occured in classic as well, but much much later.
(If it works the way I decribed above - treasure generation puts item into slot, slot attaches handle.)


I think we all know, that it could of course be fixed in existing DOS, but in a 'time-is-money' world it doesn't make sense economically, because noone will pay Larian for it. (The vast majority never ran into this problem after all, myself included after my 'private changes' ;-)
I've heard that D:OS2 is even biiger than D:OS was, so it's a pure necessity to fix this weak point.

Using int instead of small could be a possible fix which would push the limit much much further, although not solve the principal problem.
Maybe the last fix that doubled the limit was not even a change from 'small' to 'unsigned small' but was simply changing the generator's counter to start counting at -2^15 instead of 0, easy fix which would effectively double the amount of available handles. (Much easier than changing the data type. The value itself is not of interest anyway.)
If handles are handed out by a 'handle generator' it would in theory only be a matter of changing the generator to search for 'left over holes'. Any bookkeeping of 64k handles needs only an 8k bit field, which can easily be stored in a savegame and has a chance to even be permanently kept in modern processor's caches. Of course this would be dependent on handle users informing the generator about releasing previously acquired handles, which might not be the case currently and the reason for 'technical impossibility'. The save process not being able to store a generator's bookkeeping bit field might be another, maybe there are problems storing binary data in the save XMLs ;-)

Anyway, bookkeeping and reuse of handles is the only way to go, every operating system would be unable to run for a longer time if they never reused values of released file handles. Blowing up the data type for handles only pushes a limit further away but the limit will be reached at some point.


What you said about the generator - counting up, no bookkeeping ... crash on overflow ;-) - means, that we can never make any changes like deleting handles to prevent overflow, so information would be helpful, if we could find any way to prevent unnecessary handle creation in the first place.
'UsePartyLevelForTreasure' for containers comes to my mind, maybe setting that could prevent standard treasure creation and cause a fallback to old 'on-demand' creation for placed containers at least. More info could also tell modders, how dangerous blowing up treasure tables could potentially become.


Apologies for bad grammar and english, it's not so easy for us non-native speakers.

Maybe you will eventually find the time to 'dig deeper' in Belgium, I'm sure modders would do as much as they could to help soften problems if possible, but they need information. And we'd love you even more than now already ;-)

My location wasn't really a factor. I joined the previous Larian forum just after Divine Divinity was originally released, and hung around long enough that they hired me (four and a half years ago).

The changes to loot generation are probably part of the problem, but it was a change in one of the updates that increased the number of handles used and turned the 'practically unlimited' number of handles into something that could be reached with a lot of trading or crafting. When the system was implemented, it seemed more than adequate; then the game grew significantly (even then nobody hit the limit in the original release, until after the EE was out) and changes in the EE increased the demand for handles.

This did occur at least once in D:OS Classic, at a playtime over 550 hours (that was only reported after this problem was discovered and the limit raised in the EE).
D:OS 2 will not be using the same handle system.

So you're at Larian since somewhen around what, when 'the amazing spectacle that is Dragon Commander' came out ?

For me D:OS is among few best games of all times, together with a legend like Might & Magic 6 (and 7 in some ways), one Drakensang and The Temple of Elemental Evil (both the latter being way too short), oh, and not to forget something completely different, Diablo I and II (never touched Diablo 3).

At the moment, the only games I even ever play are D:OS and LOTRO.
Beating the Void Dragon was one of the saddest moments in my 'gaming history' ... because that was the moment the game was over. (An advantage of MMOs is, one can hardly ever reach that moment ;-)

A game as good as D:OS makes me passionate about it and is the reason I got interested in modding it. In order to do that I need information about how things work. A lot of info is already here in the forum but a lot is also still unknown. (Not to Larian obviously)

So I want to apologize for being so obnoxious about getting info many times, it's only because I'm so passionate about it ;-)

I started at the beginning of 2012, a while before the Divinity Anthology was released. After D:OS was released I took over email support, as well.

The documentation and support for D:OS 2 modding should be better; there is certainly an effort to keep internal documentation up to date and organized during development.

I've thought about using Osiris itself for such book keeping, with basically a DB_RecycledHandle(UUID,handle). You can get the handles even for local objects in Osiris by catching a CharacterItemHandleEvent() (these are triggered even for local objects, and you can send them to CHARACTER:NULL). However, communicating a handle from Osiris back to an item script seems to be impossible. There is no Item/CharacterSetVarItemHandle(), no CharacterItemHandleSetEvent(), and using *SetVarInteger() or *SetVarItem() does not work with handles.

BTW: watch out when dealing with handles for local objects in Osiris. The Divinity Engine crashes with a segmentation fault if you try to do certain things with these handles, presumably because in general it only sees global objects (I forgot the exact code I used, but watch out if you try to reproduce this: if this is done in code that runs automatically when your mod is started, you'll have to remove the offending code using an external editor because the Divinity Engine will immediately crash when loading your mod).

I might have mentioned that earlier somewhere. What we get as an _ItemHandle from queries or 'events' like ItemTemplateAddedToCharacter() is a handle of a complete stack and not of a single item. This might be the reason why there are no GetVar or SetVar calls because a handle does not stand for a single item.
It seems safe to query a string from a handle though because if anything is different in an actual object, it does no longer stack as can be seen with the moonstone at Cyseal North Gate, which has the PUZZLE_HiddenPerception.itemScript attached and does not stack with other moonstones anymore because of this. So I assume getting a String with ItemHandleGetVarString() is safe because all objects on a stack must have identical parameters, including their item scripts, otherwise they would not stack.
Enabling setters is probably possible but you'd have to deal with a lot of stuff if you did id:
Should ALL single items of a stack get this 'event' or just one ? If it was only one, it would have to be moved out of the stack because it was different then which would require creating a new handle for this. Then how would Osiris communicate this new handle to the original caller of the setter ?

There are some Osiris function that can deal with those stacks like for example ItemHandleMoveTo... which has an _Amount parameter and would be better named 'MoveSpecifiedAmountOffTheStackTo...'. If a handle was really a single item there would be no (INTEGER)_Amount but an (INTEGER)_Bool as a parameter, in fact, no additional parameter would really make sense ;-)

For myself I started calling these handles 'handles-of-entities-that-occupy-one-slot' because they represent what can be held in a slot in any container (including our inventories) and not a single item (can be a single one though, and always is for non-stackables like weapons of course).


I have absolutely no idea how bookkeeping of handles in Osiris could help preventing handle overflow when it's the engine that assigns handles, so how would you assign a handle to anything in Osiris ?
(If you meant bookkeeping for some very generic code to prevent it from using unusable handles, it makes more sense to me.)


Anyway, what I meant above was not bookkeping of handles in Osiris but internal bookkeeping of the 'handle generator' Raze was talking about ;-) (That's why I was blabbering about 8k bit fields and such.)
It was all about not only counting up an internal counter - like a sequence in some SQL system - but about doing a 'turnaround and restart from the beginning' once some Max(small) has been reached.
For this to work, bookkeping would have to be done, and could be done with an 8k bit field for 64k handles for example, each bit representing one possible handle value with 1 for used and 0 for free.
That would only work though if handle acquirers would ever release the handles they acquired - inform the generator about the fact that a previously handed out handle is now free to use again. My assumption is, that this releasing of handles never happens in current code and is hard to 'fix' because there might be too many places in the game (and Osiris) code that would have to be changed.

My original curiosity was if a handle could ever be reused internally by the engine when it creates new items. By that I meant the integer value of it, after an item that previously had this value as 'handle' has disappeared for good, I did not mean the item itself. Raze's answer to that was clear: no.
If internal reusability had existed, removal of items (like ItemHandleDelete()) could have helped avoiding the notorious 'crash on zoning'. As it looks, nothing but internal changes to the handle generator (and probably handle users) can help.

Do you consider changes on trader inventory generation? That's another change from Classic to EE that forces the problem, if I understood that right. Restock after one hour (real time) on trade request and after (at least one) level up on trade request.
I changed that for my mod now (after 1,5 hours and after 2 level ups) but I'm unsure how that will affect crafting for those who use to craft a lot. Would it feel too much like a restriction to available crafting resources?
As passionate craftswoman, FrauBlake: Did you run out of resources sometimes and would you even more under the new conditions?

PS: Good question to localisation. I'd like to know too.

In spite of the name, it's not crafts'woman' ;-). I only like that name because my kinsmen and -woman in LOTRO call me that, since all my characters there are female and have the last name 'Blake' ;-)

To answer your question, I bought all available ingredients whenever possible. Only the 'useful ones' of course, there are others that aren't interesting.

I don't intend to change any generation intervals. I don't intend to change treasure tables except in what traders carry. That I need to 'disable' certain crafting recipes because as far as I know there is no way to disable one. I can only take away the ingredients those recipes use. For example I want 'Scroll_Skill_Air_Blank_A' to no longer appear in the game, and I will replace all existing scrolls and skillbooks, which requires changes to treasure tables (or links in some case). Minor additions maybe, like teeth in bone piles, and I think I should remove stuff I provide better replacements for like lockpicks or disarm kits.
And if I do not forget I shall add very importand stuff that Larian forgot like for Cyseal's fish seller:


;-)

Why did you change trader generation frequencies ? Cannot be because of limited handles. (Because I am the only one paranoid about them ;-)
Changing that will mostly hit crafters because they depend on trader's ingredients, traders are the only reliable source for a lot of ingredients. (Think about the special arrows, where else would you get antlers or teeth ? Most people don't care about that, but crafters do.)


Most people played through the game without problems even before they changed the handle limit from 32k to 64k. Personally I have become paranoid after my first two play attempts resulted in that crash. Other people had that crash too.
Gen. Raze said that the problem even occured in classic ... after some 550 hours of gameplay. (Okay, only reported once, there might have been more cases.)
I was very far away from those 550 hours when I had my crashes, don't know, could have been 80 to 100 hours.

So put together: the problem was non existant in classic and occured in EE with more than one person reporting it (myself and others NOT reporting it).
The only thing I can think of that could have a big influence on this is the changed treasure generation, because, what else ?

So ... treasure must be pre-generated. Otherwise it would change after loading.

Although it's probably pre-generated, traders receive the items they sell when they need them, I had ItemHandleAddedToCharacter() events for all their 'generated' treasure, so the treasure was added when I started trading. I haven't tested if 'save scumming' works with traders, if it does, it is a real generation on demand for traders. If not, pre-generated treasure sits somewhere in the background waiting to be 'claimed'.

I think treasure is pre-generated every time you level up in the current area, every time you enter a zone and for traders every time after new trader treasure has been generated. (To have something sitting in the back for the next generation and prevent the end of the world: save scumming. It's probably a normal treasure but the Osiris call GenerateTradeTreasure() triggers the immediate creation of a new one.)
Treasure MUST actually be created on every levelup, because no one would like to open chests, barrels or crates with loot of the level it had when you first entered an area. You can easily get to Hiberheim at very low level if you visit Wally ... or Willy ?. But you would not want level 5 loot in all Hiberheim. You also enter Luculla Forest a long time before you actually really do something there, because you need to enter it to go to Hiberheim. If you open a container in Hiberheim or Luculla though, it's around your level and not much higher I think. And finally, you stay in Cyseal for a while without ever leaving it, you could open ANY container any time, loot should be at an appropriate level and not level 3, the last time you left Cyseal and came back after the second star stone.
The only exceptions to this migth be containers with a fixed level because that treasure would not need to be recreated, and containers with 'UsePartyLevelForTreasure' might use on-demand creation. (The latter is doubtful because it would allow for the single worst thing that could ever happen anywhere on this planet: save scumming to get better loot. If it were the case though, I'd know exactly, what I'd do in a mod ;-)
Mob drops seem to use the old on-demand creation from what I've heard. (And maybe the 'custom treasure thing' somewhere in _Trade.txt in Osiris too.)

Since it could be quite an effort to differentiate between 'categories of treasure generation', maybe all treasure (except mob drops and 'custom') is even always generated at the above mentioned occasions.
Could be that treasure in an undiscovered area within a zone is first created when the area is discovered to 'soften the problem'.

If I am right and with treasure pre-generation also comes handle acqusition, it means that the change from 'on demand' to 'pre-generated' treasure in a system without any 'handle handling' might have been the single most stupid change from classic to EE ;-)
I never understood the motivation behind it either. If someone wants to save scum, why not, does not hurt others. After you load the game for the tenth time, it gets old and you go with what you have.


I could speculate a lot about how things work, but in the end, only General Raze can give us answers.
(... or confuse us even more of course ;-)

Oh, sorry, man!


You would have to exchange the file in the pak file. Should work, shouldn't it?


One player missed scissors. How to cut the hair otherwise?


You're not alone (if that's calming for a Paranoid). Since the first time I heard of this precarious matter I developed some pretty fears myself. When will that ticking bomb blow up? And how to handle ticking bombs without, oh God, using handles!

I'm not sure about the relevance of trader restocking. Depends on playstyle (how often you request trades), so the influence of EE is more indirect. It encourages massive trading and crafting much more than Classic did (only level, not time difference relevant). And it forces restocking (more than Classic) when you just come back to buy an item you didn't buy last time (and Classic only refreshed empty slots, if I remember correctly). Well, I see the point: I would hurt those I try to protect.
But there's also a non-handle-related problem I have with the trading system: in Classic I often visited other vendors to compare items. Now I always risk to lose the item (due to time related refreshment) if I don't buy it immediately. And trader assortments (speaking of armor and weapons) become too exchangable with such a high frequency. And third problem: trader items are just too powerful. So the solution would be to make changes on frequency (best only for armor and weapons, not for ingredients) or to change treasure tables (of vendors and/or enemies) in the last case.


Yes, that's the big change in EE, to prevent save scumming. To a very high price...


I almost said: Yes, Sokrates, it cannot be otherwise! Now I see clearly: items must be pre-generated on every levelup. Otherwise you wouldn't be able to find legendaries since they require a certain level to drop, as far as I know. And you definitely get legendaries (at least in most cases) after a while on the Cyseal map.
So the difference of quality between trader items and looted items only comes from different treasure tables?


I tend to agree.

EDIT: Do we know what CharacterMoveWeaponsToContainer and ContainerMoveAllItemsToCharacter does in that regard? Does it create handles for all weapons/items?

And do ItemTemplateIsInPartyInventory, ItemTemplateAddToCharacter and ItemTemplateRemoveFromCharacter deal with handles? (I'm scared...)

No offense taken, I wouldn't use that name if I had a problem with it ;-)


Yes, but new mods don't work the way the old ones did in the pre-editor time. I do not want anybody to have to do the 'rename MainLSF.pak' thing.
And since recipes can have multiple output, adding a new one might only add a new output but not disable the old recipe. (Is that understandable ? I mean of course severaly recipes with identical ingredients and identical requirement.)


In classic times, I even created an icon for scissors. The problem is that there is still no way to implement hair-cutting ;-)
Well, maybe changing visual resources could be done but that is currently beyond my abilities ;-)

The reason I want to add this TT is that I love this line from the fish seller so much:
"How many crab claws would you like ? Three ? Eleven ?"
I think it's hilarious.


I was never a big fan of 'playing the economy game', neither here nor in any MMO I played. I personally don't care if it's 'overpowered' or not and I don't care if other people 'take advantage' of it or not.
Of course there are always complaints of players if something seems easy. Telling people that something in a game - like 'abusing' traders - is not mandatory for them to use doesn't help, they won't have their peace until everyone has inconvenience.
This is might also be the reason for the whole treasure generation change: catering to this group.

I used traders mostly for crafting ingredients, less for anything else. (The air skill seller and the quartermaster in Cyseal were the ones I visited most, because they sell ingredients ;-)

One could count how many different items a trader has and how many restocks it takes until a relly important number is reached. If traders create treasure only 'after demand' (i.e. immediatley after getting their last pre-created treasure) the number would not be very high, unless ALL traders are always visited, which only a tiny minority of players does.
We also don't know, if traders get their complete inventory restocked or only a 'delta'.

One possible solution to soften the problem is to introduce one trader who does not sell anything but has lots of money. Most people use traders only to sell stuff and they have to visit all of them because they want to sell more than traders can give them back in money. With every trade start there is a good chance for a new treasure generation. Removing 'useless' stuff from traders helps too. Traders for example really don't need to carry buckets or cooking pots, because there are enough around everywhere.
Of course, with every change, the 'role playing problem' arises.


As I said, still don't understand why save scumming is a problem, one does not have to do it ;-)


There is a chance though, that only containers in currently discovered areas are 'refilled'. But they would all have to be 'refilled'. There is also a chance, that treasure is even generated after a container has been cleared, but immediately thrown away, since the container is supposed to be 'empty'.
Seems stupid but some performance or efficiency thing in the engine might require it.
(Without info, I believe, a whole area is 'treasured' on the mentioned occasions, just because of internal 'engine efficiency', not having to check a lot of external stuff.)


Should not because you move stuff that already has handles.


Depends.

The IsIn() calls cannot do any harm because they create nothing but query only for existing values. (would be more than weird if they did.)
Same for the Remove(), how would removing require a new handle?
AddTo() calls must use a new handle because they create something, which is not different to treasure creation or crafting. There is a chance that no new handle is used if we create a stackable item and a stack of that item is already present (and not full) in the target container. Larian knows that.

As I mentioned - probably above in this thread - handles are no item handles but 'slot handles' or better 'handles of objects that occupy one slot in a container'.
That is the reason why the query GetItemHandleForItemTemplateInInventory() (or similar ;-) does not return a tuple but a single entry for a stack of items.
Since weapons and armor are not stackable, crafting them for sale is real poison in the whole handle problem context.

If I think about it, an essential change I made last year after my two crashes was to no longer require crafting of weapons to make money, instead I crafted a stackable item from bars to sell and I always had one or more of them in the inventory. That suggests that my creation did indeed not use any new handles because a newly created object was added to an existing stack which already had a handle. New handles were only created if I sold the complete stack and started a new one afterwards or if I only sold part of my stack to a trader I had not sold any of those items before, because then the trader needed to get a new handle for this new stack because he/she could not add it to an existing stack and could not take over my existing handle, because that was still used for what was left in my inventory.
In my first two playthroughs I had crafted tons of weapons to sell them.
(Another change was a massive increase to stack sizes. Last year I thought that traders inventories had some overflow that caused the crashes, I had no idea about handles yet. Coincidently, my changes affected pretty much the handle problem.)


But all this is pure speculation, I only try to understand how things work.
As I said, Raze is the man and our only connection to real info, but I guess he is busy enough with D:OS2 ... and support of course ;-)

We probably shouldn't worry too much about this, as long as we don't know what's really going on.
We speculate about how we could push the limit further away, that might be worthwhile.

But I was thinking about characters. They have handles too. If they take the handles from the same pool as items, then spawning or summoning characters alone could cause the crash, although it might take a lot longer than 550 hours ;-)

My 'field bed skill' also spawns an item, maybe I should no longer have it talk, because that might lead people into over-using the skill just for fun ;-)

Since this seems more about traders now :P, maybe you guys know the answer to this one:

So if vendors refresh based on time, do random NPCs as well? If Random NPCs do not refresh based on time, what differentiates the vendors so that they refresh (via Osiris I assume).

It's mostly about our paranioa with running out of handles ;-)

So ..... one could ask, are there really random NPCs ? Are there really traders ?

My answer would be no, the 'traders' only have much bigger treasure tables, that makes them different, they react differently to theft (because Osiris tells them to) and they claim item ownership in a certain area (also because Osiris tells them so).
Otherwise they are not different at all from the 'randoms', so yes, the randoms have exactly the same treasure generation.

The random NPCs don't matter that much though because usually they use very small treasure tables like 'GenericNpc' and there have to be very many treasure creations to really matter. From what I've heard, enemy mobs still allow save scumming, so they must be exempt from normal treasure pre-creation.
(I don't know how gold is handled, but gold uses a root template too, but that must be an implicit one, because we don't see it anywhere in Equipment.txt. I have absolutely no idea, where that comes from but I know that I can remove all my gold with a CharacterAddGold( -Amount ) and but I can also remove it by moving the handle of it's root template somewhere else. I can add gold with CharacterAddGold() but I can also add gold with ItemTemplateAddToCharacter( "LOOT_Gold_A_Big_A_.....whatever", _Char, _Amount ), so gold is an absolute enigma to me ;-)

As mentioned, I'm not sure if pre-creation works for trade treasure generation too or if it is only done for real containers. My guess is, that trade treasure is also pre-created.

The two code pieces in _Trade.txt that check if treasure should be generated are



for the 'levelup' recreation and



for the 'after 1 hour' creation

I cannot find anything in the code that suggests that traders are handled in any way different. They are only normal NPCs after all.

So it's possible to disable the regeneration just by commenting out those scripts then?

Also you could probably modify the script and create a list of predefined traders as DB query so only NPCs that are meant to be vendors regenerate. Yeah they don't use as many handles but those 1-3 items on each NPC might add up over the course of the entire game.

Though for some reason I thought there already was a trader DB somewhere, which is why I thought generic flavor NPCS like those around the docks might not regenerate based on time/level. I'd look at what calls GenTradeItems and see which _Trader's get passed.


For the mobs, a lot of them are local characters so that makes sense.

Mainly I ask because I use the TimeOfDay in Dunamis for my Day/night cycle and the traders behaving strangely can be explained by me accelerating the gamehours.

The IF RequestTrade(_Player,_Npc) event calls StartTrade() which then calls GenTradeItems() which might create treasure.
Osiris does not handle mob drops, the engine does. So global/local does not matter, bosses drop random as well from what I've heard, and all of them are global.

The time thing is done with a recursive timer in _AAA_FirstGoal.txt. (They still have a comment expecting the engine to give them an event ;-)

Yeah, disable treasure regeneration for non traders might be way to push reaching the limit further away. But it does not make sense to take steps if we don't even know the slightest bit of how it really works behind the scene.

As we discussed though, traders can only become a problem if one is really very long into the game. There might be a way bigger problem with unopened containers.
(All vases in HOM for example have 'UsePartyLevelForTreasureLevel' set, so it might be tempting to wait until you loot them, resulting in a new creation every time you enter HOM.)

Really, they should just have left treasure creation alone in the first place, this whole save scumming is really not the end of the world.

Since they didn't, I now get paranoid if I should let my mattress talk, because I spawn it, and this probably uses a new handle every time ;-)

... and as said above, characters have handles too, and we don't know if those handles come from the same handle pool, so we remove summon skills ... ;-)

Sorry to ask again: so we speak of 'handles of objects that occupy one slot in a certain container/character inventory' and not of 'handles of objects that occupy one slot in any container or character inventory'? So is the handle for a stack of objects or a slot with objects? And: is a new handle created when I move a stack of objects to another character inventory or container?


It seems Osiris iterates all characters when the goal _NPC_Stats is initiated and starts a process to add them to DB_Trader if certain conditions are fulfilled:



It doesn't except trading NPCs from trade item generation.

Side note: pickpocket requests also start the GenTradeItems process.