Wonder how accurate this is ...

[Aeni]

http://www.fleckz.com/ah/?sid=1&cid=1

Note, this is only for two server worlds, as I'm assuming whoever created this has accounts on those two servers and was motivated enough to start up something of this magnitude.

Do you think that SE would ever release an official online version of their AH? That would be interesting, to be able to buy on the AH without having to log in ... maybe even sell too, if the items are already in inventory (i.e., your mule)

[Tan'o]

Re: Wonder how accurate this is ...

http://www.fleckz.com/ah/?sid=1&cid=1

Note, this is only for two server worlds, as I'm assuming whoever created this has accounts on those two servers and was motivated enough to start up something of this magnitude.

Do you think that SE would ever release an official online version of their AH? That would be interesting, to be able to buy on the AH without having to log in ... maybe even sell too, if the items are already in inventory (i.e., your mule)

From what I've heard, the data on both servers is completely accurate.

About SE releasing an off-line version, it's a great idea, but it doesn't take that long to log in. Besides, the last thing any of us want is more maintenance for broken AH/search function. It's a good idea but just a little unneeded.

[Macht]

Re: Wonder how accurate this is ...

You know how much work that takes to keep up on all those prices? There's no way you collect all the items prices in a timely manner to make much a difference unless you were to access the AH server directly. Even then accessing a server directly to pull out data like this from it that's no better then releasing a DDoS attack, it does explain why the search and AH have crashed a lot lately.

[Omni]

Re: Wonder how accurate this is ...

technically speaking, i dont think releasing an web version of the ah would be much of a problem. all of the AH is tied to a database (ie: sql for example) all they would need to do is create a web app to read from the same db. all transactions would be real time and accurate b/c it would be reading from the same db the game uses. however, the only problem would be cost. this would cost TONS of bandwidth along with additional infrastructure. This also makes it a lot easier for people to attack and disrupt gameplay. if we lived in a happy happy world, i think it would be awesome. you could check on your auctions from work, your mobile device, etc. however, putting things on auction might be tricky... maybe your moogle can do it for you? it would also be intereting if you can log all your sales. track your sales and buys of a certain item. you could also enter how much you paid for them and track profits/losses etc. super doable, is it viable cost/return wise? probably not.

edit: thinking about this some more, maybe they dont have to let you access the db real time. maybe they would allow you to export a file that would contain all your recent sales, in any standard format (ie: comma separated, etc) then you could upload that to their web-ah. you can track your sales trends and stuff.

[Macht]

Re: Wonder how accurate this is ...

technically speaking, i dont think releasing an web version of the ah would be much of a problem. all of the AH is tied to a database (ie: sql for example) all they would need to do is create a web app to read from the same db. all transactions would be real time and accurate b/c it would be reading from the same db the game uses. however, the only problem would be cost. this would cost TONS of bandwidth along with additional infrastructure. This also makes it a lot easier for people to attack and disrupt gameplay. if we lived in a happy happy world, i think it would be awesome. you could check on your auctions from work, your mobile device, etc. however, putting things on auction might be tricky... maybe your moogle can do it for you? it would also be intereting if you can log all your sales. track your sales and buys of a certain item. you could also enter how much you paid for them and track profits/losses etc. super doable, is it viable cost/return wise? probably not.

Yes, it is SQL but I have my doubts that SE indexed the player names in the databases and running queries off of fields that are not indexed greatly impact the SQLs speed and in turn also effect the databases performance. That coupled with what you and I are in agreement with bandwidth it would just murder the Search and AH Server, which lately I've noticed it having a lot of trouble. This data here probably explains why.

Outside of that depending on the Server as well. At my work place the AS400 server has a flaw with Pass-Through Queries, if the receiving end disconnects while the query is executing the query becomes a runaway and starts eating up resourcses exponentially. If the query isn't caught and executed quickly it can corrupted the server and cause for us what would be a 3 week maintenance to get it running again.

EDIT:

To make it more obvious of what I'm stating if these guys are accessing the server directly to pull that data with this "tool" they state then they are playing with fire. They could end up really f****ng up the database. The only people really that should be messing with those databases is SE and only SE.

[Omni]

Re: Wonder how accurate this is ...

yea so this is more likely than not viable but it would be really interesting to see. it would take game play to a whole other level i think.

bandwidth is a pain but, im sure they run their own data center and they get bandwith pretty damn cheap. if you buy in advanced, say, 100gb of bandwidth a month, its way cheaper than buying 10gb a month and paying extra for the 90gb. but with more bandwidth, theres more infrastructure. i can only imagine the magnitude of their data center. buying racks full of switches, load balancers, firewalls is just a bunch of cost that needs to be passed on somewhere, most likely the end user. which i think wouldnt go over too well with a lot of people.

i really want to know how they would be pulling that data? i mean how can they access the data w/o doing anything invasive? you wouldnt be allowed through the firewall, let alone have read access to the sql db.

you are right about managing your ah from the web. that would be a nightmare in terms of support. it just provides another point of failure.

i'd would love to see this happen somehow. maybe not in ffxi but in another game. you can tie your game account to a web account and make small changes via the web or better yet, what about a mini trainer version that you can export to a psp? maybe you would have to be in your mog house for it to work, since all other areas are open to other players, but you could do such things like crafting. the data would be saved on your memory stick and once you upload it to the game, you can have your newly synthed items in ur mog ready to be put on sale or whatever.

again, thinking too ideally. this would totally open up room for hacking items. further killing the ah.

[Aeni]

Re: Wonder how accurate this is ...

Here is how I think they are doing this.

Basically, the connection to the database uses SQL queries. It is then a matter of a hit or miss type of chance using various languages until you come up with something that SE's database can understand. For instance, using a Python script and moving on to MSSQL commands if that fails ...

For example, a RUN (connection name) AND SELECT (all recordsets) FROM (table name) You can probably get the name of the tables and other such things using a Data Viewer. Such things are easily found on the 'net now.

I don't think this thread is the appropriate thread to discuss this issue in detail since it might violate POL ToS and/or FFXI Online's policy, so I'll just stop here. However, knowing that you can view recordset, if someone accidentally happens on a admin account, then that's when the real problems will arise ...

[Omni]

Re: Wonder how accurate this is ...

hmm... but you still need to get past a firewall, which would not allow any connections on 1433 from any ip otherwise listed in the firewall. let alone, you wouldnt be able to hit the db directly since i doubt any of them have external addresses. all data results are being queried from an app server right? and the results displayed to us via the app server on our client side.

i wonder if they are using the game somehow to make these queries?

[Macht]

Re: Wonder how accurate this is ...

Here is how I think they are doing this.

Basically, the connection to the database uses SQL queries. It is then a matter of a hit or miss type of chance using various languages until you come up with something that SE's database can understand. For instance, using a Python script and moving on to MSSQL commands if that fails ...

For example, a RUN (connection name) AND SELECT (all recordsets) FROM (table name) You can probably get the name of the tables and other such things using a Data Viewer. Such things are easily found on the 'net now.

I don't think this thread is the appropriate thread to discuss this issue in detail since it might violate POL ToS and/or FFXI Online's policy, so I'll just stop here. However, knowing that you can view recordset, if someone accidentally happens on a admin account, then that's when the real problems will arise ...

Ok, not exactly what the focus was but close. The point I was making is someone doing pot shots at a database like that can have some serious problems. Mainly because they are taking pot shots at databases were they have no clue of it locking methods, transfer methods, it's indexes, and also ignoring that their actions could end up triggering something similar to what happens on my companies AS400 system.

I mean the worst possible problem that could occure on our AS400 system, and almost happened because none of the user that access it knew it would be a problem. Hell IT didn't even know the problem could occur and was dumb luck they found it literally minutes before it crashed the AS400. That error when they calculated out the damage it would of cause would of been 1 month were our corporate office would be dead in the water as far as operations. 1 month! That's a damaging blow especially done during our peak season which only consists of 1 month.

That's why I find a site like that disturbing. If they were actually able to access the database and also were taking these blind shots at it. They were playing with fire one that could of left SE with angry customers for who knows how long, you know for certain if something like that happens SE would be demanding heads to roll.

[Caspian]

Re: Wonder how accurate this is ...

I wasn't getting anything current on the site though. Most recent sales I could find on anything were from Nov. 1st. Only about a month and a half behind. So obviously he's manually going through and writing down everything. And he's about a ways behind on updates.

[Aeni]

Re: Wonder how accurate this is ...

Omni,

If your client is able to retrieve data, then there is no defensive barrier per se. You can disguise any request made anywhere outside of the client by pointing it to the right ip address and wrapping that request in your POL ID. That POL ID is tied to an FFXI account number (Your registered CD key basically) and that is how your request is legitimate.

Macht,

I know what you are implying. I did not in any way condone the behavior of the individuals who do this sort of thing ("tapping" into SE's servers to extract data to be polled outside of the client interface) What I only brought up was a possibility that SE could "enhance" your gaming experience by providing a way for a player to be able to participate in the game outside of the client (e.g., if you were on vacation for 9 days and you have 80 stacks of shihei you're trying to get rid off on your mules from crafting and want them all liquidated while you're not able to physically log in through the client)

Think about it, if SE initiated this and provided an alternate way of doing things without having these reckless individuals hacking up the system to achieve the same goals, then they would definitely avoid a disaster like the scenario with your company that you outlined above.

[ibroyles]

Re: Wonder how accurate this is ...

I wasn't getting anything current on the site though. Most recent sales I could find on anything were from Nov. 1st. Only about a month and a half behind. So obviously he's manually going through and writing down everything. And he's about a ways behind on updates.

Correct. One of these is on Titan server, and I think they just have 2 or 3 guys going through and compiling this data by hand. If you notice, some items are even missing from the database. Also, quick selling items are often omitted.

I originally saw a thread about this on a Titan server forum on Allakhazam I believe about two months ago. They said something about doing it manually by hand currently, but "after the next windower update" they would be updating their methods of data retrieval. So whatever that means...I don't really keep up w/ Windower updates as I play on PS2.

[Caspian]

Re: Wonder how accurate this is ...

I can't imagine SE ever going online with the AH and making it possible to buy and sell w/o doing it with your char ingame. If they did though it would make things rather interesting. Half the time I'm on, it either too late to do anything worth my time, or I just dont have the inclination to do so. I usually just end up camping the AH for antican pauldrons for desynthing. Nights at work, like tonight, when I have nothing to do for hours on end would make it real interesting. I could just hang out at work amassing pauldrons for desynthing when I get home. 8 hours of AH camping would leave me with a lot of pauldrons and eventually a lot of money.

[Aeni]

Re: Wonder how accurate this is ...

Not only the convenience, but wouldn't you agree this will move items a lot faster on AH, not leave too much of a history trail and might even cut into profits of gil sellers, if the crafting community at large are able to tap into the AH without having to be at home and thus artificially "expanding" play time without ever having to log in through the client period. Less characters in AH areas means less lag. Less lag means better overall performance. There's a lot of players out there, the silent majority I call them, that really do not get in the game much at all unless it is for crafting or checking on their plants. They do not buy gil online and take the longer (in time) route of accumulating gil. When they have enough gil to purchase a desirable item or enough for food, then they spend a bit more time in the game, either exp hunting or what have you with everyone else ...

[fleckz]

Re: Wonder how accurate this is ...

This site does not access SE database directly. The data simply comes from parsing logs in the chat window that contain the price history of items. With the data, I made my own database to serve the FFXI community. The data isn't updated real-time (you could probably tell). I check item price histories when I get the chance.

The AH and sea functions had problems before we started our project. I don't like taking the blame for that, but I do believe it is players hammering the AH for their own personal gain or senseless DDOS. I don't want to speculate what the problem is but SE will deal with it.