How did this get graded 85+?

[Link | 2,709]

1 hour ago, Gloves said:

@jonebone calling it subjective is to me lazy. There of course could be very specific, predictable rules put in place that could be followed exactly and perfectly. 

bingo

[jonebone | 1,307]

6 hours ago, joshdose said:

I hate to keep feeding the fire on this thread, but I wanted to 1) take a minute to defend VGA. In total, I have personally sent in about 60 or so games for grading. I’ve learned a lot about what to expect from them with each submission. I also learned to request the add-on grade report for games I was extremely anxious about. You can learn A LOT from these grade reports. It’s the best way to gather insight into what they look for in grading different types of games. In that regard, I have found enough evidence to support the idea that they definitely have a set of strict and consistent standards. Can you blame them for not publicly giving out their grading system protocols / procedures? Unless you purchase a detailed grade report, you won’t know why a game you thought was gem mint scored an 85 or a game you think is unworthy of  gold because there is some flaw on it that can be easy to spot. 3) I’m the only one here, other than VGA, that actually held the CC in my hands and studied it under magnification lighting. 2) The tiny tears in the spine are the only issue I remember this game having. A big factor that goes into grading of jewel case games is yellowing. Most people don’t even know to look for this and won’t ever notice it, unless you have the game unslabbed and compare it to another game that is in fact gem mint. It will drop your grade big time and you won’t even know why. Another thing that’s looked at closely is the centering of the sticker labels on top of the game. A lot of PlayStation games had off-center labels that most people don’t notice or know to even look for. These are just a couple examples, but there are more, a lot more, referenced potential flaws that can be ascertained from VGA’s methods by way of grade reports. In the case of this Chrono Cross, I did not purchase a grade report, but I can attest that it was in fantastic condition overall, before sending it in. (Other than the obvious tears on the side). I mean, I had other sealed Chrono Crosses in my hand that I was able to compare to. Most of them had slight yellowing and / or slight UV fade of the artwork. Both of which would have gone un-noticed by me, had I only had the one copy and no “prettier” or “uglier” ones to compare against. I was quite confident that one was the best overall. If it’s something you genuinely care about, you can instruct them notify you of the grade before slabbing it, if you choose to have it slabbed, inquire about the detailed grade report. There will be no mystery that way.

1) I defend VGA too, I own a lot of their games and have graded many myself.  My rule of thumb with both grading services is that I agree with about 80% of the grades and find roughly 10% overgraded or 10% undergraded.  You don't have to agree with every single grade you see to believe in a company.

2) Also, you are forgetting the HUGE 1" (at least?) deep scratch in the back plastic.  Let's be realistic here on PS1.  If you want a gold you have to find a game with no cracks or drill holes and then it all comes down to wrap condition.  You can get by with some extremely light scuffing if it is light and maybe a tiny tear / corner poke or perhaps two.  This game has multiple tears and that scratch is not a scuff, that is damage like something was raked across the wrap (fingernail?).  As opposed to minor light scuffing that you don't notice unless tilting in various angles under the light.

So if "sticker placement" is perfect and even if the rest of the game surface looks like absolute pristine glass (as a Mint game would appear), still no way it is better than 85 for me personally.  Just because I do not agree with their grade does not mean I am bashing them, it means I consider myself to have an informed opinion and I find that my opinion disagrees with the opinion they have rendered here.

And as a caveat, I do agree that you can only accurately assess a game by holding it in hand. So your point 3 does have merit and I do tend to defer to the opinion of an expert who has held the game in hand.  However, even if the rest of the game is absolutely perfect I see too many flaws to consider this a Gold 85+ game personally. 

[jonebone | 1,307]

6 hours ago, Gloves said:

I completely disagree that it can't be automated. In fact grading video games (or anything else) is something that machine learning would be absolutely perfect for. 

@jonebone calling it subjective is to me lazy. There of course could be very specific, predictable rules put in place that could be followed exactly and perfectly. 

How bad is a hole in the wrap? Let's say it's based on size and position. Let's say a 1cm long cut constitutes a -5 to a grade. Is that cut on the front? Multiply the negative by 1.1 if on front, 1.05 on back. Just an example with simple, made up numbers, but it's a start and far better than giving the grading folks the benefit of the doubt as the experts.

I don't know that it will ever happen, but here's a free idea for any smart folks out there - get your ass on machine learning and set it to making a grading system for games & comics. My money's on you'd straight up kill VGA and Wata in one fell swoop, and if neither of those companies are looking into machine learning they are being foolish IMO. We live in an age of technology and it IS there RIGHT NOW; this is possible TODAY. 

I'd trust a machine that's been set to the specific task of grading games over a human any day.

With all due respect, how many graded games do you own exactly?  How many have you submitted yourself?  I see you occasionally talking about them but get the impression you don't own many if at all.

The reason why I'm asking is that you are speaking from a theoretical perspective, not a practical perspective.  Sure, in theory grading could be automated just like driving could be fully automated by self-driving cars.  

The problem lies in that you are looking like this as a math problem when really it is like grading an English essay.  Can you have a machine grade an English essay or do you think that it is important to have a professor read it and render his subjective opinion?  Machine would catch easy stuff like punctuation and grammar but would not accurately assess the body of work.

Same thing here.  It's easy to make a rubric based on number of tears, length of a crack / crease, length of a scuff, etc.  But then you get to more subjective wear, yellowing, humidity damage, sticker residue or remnants, warping, manufacturing damage vs. shelfwear, surface abrasions, etc.  

Then, even if you find a way to program every single possible type of wear (which you won't know until you see a new game that has a specific issue you've never seen before, which will happen) you have to somehow prioritize the wear patterns (likely with a human).  Is yellowing worse than a tear?  What about two tears?  Is a tear worse than a crease?  Is a price sticker even a flaw?  What about a loose disc on PS1, is that a flaw?  Etc.

And then the grader's job is to subjectively assess the total game based on pointing out all of the visible discrete flaws.  That's where you have the human grading the essay because they look at the entire piece as a body of work.

So ultimately, sure, maybe you can use automation to assess the severity of certain flaws, but you still need a human rendering an opinion to grade the game.  Either in programming the AI to rank the flaws or assessing the game at the end.  The human is a non removable part of the equation.

And lastly we are also dealing with discrete intervals, not a continuous range.  You aren't getting a grade of 9.083415 based on a computer output.  For example, you'd have an 8.5 which maybe captures 82.5 through 87.5 while higher would go to 9.0 and lower goes to 8.0. And those borderline 87.5 games are the entire point of this discussion.  Do they bump up or bump down and it depends on the both the ranking and the severity of those flaws.

TLDR: No, you can't automate grading.

Edited March 10, 2020 by jonebone

[Startyde | 175]

It's funny, if you were going to automate anything grading wise, it would probably start with BGS or PSA centering of cards, something that should not have one air of subjectiveness and even they don't do it (can't tell you how many times I've seen off-center 10).

TLDR: Jone is right, robots aren't going to do the dirty work for a long long time. The best thing to hope for is consistency. I love VGA, that will never change, bit also as Jone said, this 85+ would fall out of my standard deviation of agreement had it been mine. Then again I wouldn't have subbed a PS1 game with tears so it's moot.

[Gloves | 11,771]

2 hours ago, jonebone said:

With all due respect, how many graded games do you own exactly?  How many have you submitted yourself?  I see you occasionally talking about them but get the impression you don't own many if at all.

The reason why I'm asking is that you are speaking from a theoretical perspective, not a practical perspective.  Sure, in theory grading could be automated just like driving could be fully automated by self-driving cars.  

The problem lies in that you are looking like this as a math problem when really it is like grading an English essay.  Can you have a machine grade an English essay or do you think that it is important to have a professor read it and render his subjective opinion?  Machine would catch easy stuff like punctuation and grammar but would not accurately assess the body of work.

Same thing here.  It's easy to make a rubric based on number of tears, length of a crack / crease, length of a scuff, etc.  But then you get to more subjective wear, yellowing, humidity damage, sticker residue or remnants, warping, manufacturing damage vs. shelfwear, surface abrasions, etc.  

Then, even if you find a way to program every single possible type of wear (which you won't know until you see a new game that has a specific issue you've never seen before, which will happen) you have to somehow prioritize the wear patterns (likely with a human).  Is yellowing worse than a tear?  What about two tears?  Is a tear worse than a crease?  Is a price sticker even a flaw?  What about a loose disc on PS1, is that a flaw?  Etc.

And then the grader's job is to subjectively assess the total game based on pointing out all of the visible discrete flaws.  That's where you have the human grading the essay because they look at the entire piece as a body of work.

So ultimately, sure, maybe you can use automation to assess the severity of certain flaws, but you still need a human rendering an opinion to grade the game.  Either in programming the AI to rank the flaws or assessing the game at the end.  The human is a non removable part of the equation.

And lastly we are also dealing with discrete intervals, not a continuous range.  You aren't getting a grade of 9.083415 based on a computer output.  For example, you'd have an 8.5 which maybe captures 82.5 through 87.5 while higher would go to 9.0 and lower goes to 8.0. And those borderline 87.5 games are the entire point of this discussion.  Do they bump up or bump down and it depends on the both the ranking and the severity of those flaws.

TLDR: No, you can't automate grading.

Half or more of the arguments you make sound to me to be in favor of automation, you seem to be just ignoring it. To answer your question, I own zero graded games, and only one sealed game which I plan to keep sealed, and may consider grading. I'm not in a rush however to grade as long as the grading companies remain as imperfect as they are, because to me it's almost pointless if it's subjective. 

Consider me unswayed, and still disagreeing. This could absolutely be automated. I can tell with an algorithm whether, for instance, a tear is over a face vs over some writing vs over a mountain. 

Obviously it has to be programmed by humans. And sure, consider a final handoff to a human to verify the machines work. You're making odd arguments about rounding though. There would BE no "is it an 87.5 or should it be a 90?" if we had decimal place accuracy. It sounds like you're saying accuracy is... A detriment? 

This isn't "how does this make me feel" it's "how not perfect is this game?". 

[GPX | 1,369]

1 hour ago, Gloves said:

Half or more of the arguments you make sound to me to be in favor of automation, you seem to be just ignoring it. To answer your question, I own zero graded games, and only one sealed game which I plan to keep sealed, and may consider grading. I'm not in a rush however to grade as long as the grading companies remain as imperfect as they are, because to me it's almost pointless if it's subjective. 

Consider me unswayed, and still disagreeing. This could absolutely be automated. I can tell with an algorithm whether, for instance, a tear is over a face vs over some writing vs over a mountain. 

Obviously it has to be programmed by humans. And sure, consider a final handoff to a human to verify the machines work. You're making odd arguments about rounding though. There would BE no "is it an 87.5 or should it be a 90?" if we had decimal place accuracy. It sounds like you're saying accuracy is... A detriment? 

This isn't "how does this make me feel" it's "how not perfect is this game?". 

You can make a subjective matter more objective, but it’s actually impossible for a subjective matter to be fully objective.

Jonebone’s analogy is a good one regarding English and Maths. Maths being more objective and we all can agree on what’s a perfect score and what deserves a 90 or 85 score. With English, it gets murky judging “non-perfection” and what warrants a 85 vs 90 score.

I can chime in with another analogy, which is probably more relevant to the grading of games - the judging of the Miss Universe competitions. We all can agree no one on Earth is entirely “perfect”, but can we put in an objective formula and let robots replace the human judges? 

Even if the robots have advanced technology to detect all bodily features for assymetry, height to weight ratio, skin tone, would you be confident your opinion would align with these robot judges of automation?

 

[Gloves | 11,771]

14 minutes ago, GPX said:

You can make a subjective matter more objective, but it’s actually impossible for a subjective matter to be fully objective.

Jonebone’s analogy is a good one regarding English and Maths. Maths being more objective and we all can agree on what’s a perfect score and what deserves a 90 or 85 score. With English, it gets murky judging “non-perfection” and what warrants a 85 vs 90 score.

I can chime in with another analogy, which is probably more relevant to the grading of games - the judging of the Miss Universe competitions. We all can agree no one on Earth is entirely “perfect”, but can we put in an objective formula and let robots replace the human judges? 

Even if the robots have advanced technology to detect all bodily features for assymetry, height to weight ratio, skin tone, would you be confident your opinion would align with these robot judges of automation?

 

Again, I'd say you're using arguments that would be perfectly valid on the other side. Yes, of course a Miss Universe pageant would be better judged by robots; I'd trust that a machine algorithm doesn't make racist judgements over a human judge, for instance. My opinion is irrelevant, flawed, and subject to my own quirks. I might not agree that the winner is the hottest to me, but she'd certainly be the most perfect. 

I don't think subjectivity belongs in grading, that is my stance. It's always somewhere between perfect and a pile of ash, and everything in between is entirely quantifiable. 

[GPX | 1,369]

4 minutes ago, Gloves said:

Again, I'd say you're using arguments that would be perfectly valid on the other side. Yes, of course a Miss Universe pageant would be better judged by robots; I'd trost that a machine algorithm doesn't make racist judgements over a human judge, for instance. 

I don't think subjectivity belongs in grading, that is my stance. It's always somewhere between perfect and a pile of ash, and everything in between is entirely quantifiable. 

Don’t get me wrong, you can be objective on grading but there will ALWAYS be a component of subjectivity. That is, your opinion will not always align with the graders. 

In the Miss Pageant analogy, don’t forget the robots would be programmed by someone with their own subjectivity on what would warrant a 0.1 deduction as opposed to a 0.2 deduction. 

So effectively, 

- human judges are trying to be objective with their subjectivity

- robot judges are being objective but reliant on the subjectivity of the programmer (for point deductions).

= same diff!

[Gloves | 11,771]

2 hours ago, GPX said:

Don’t get me wrong, you can be objective on grading but there will ALWAYS be a component of subjectivity. That is, your opinion will not always align with the graders. 

In the Miss Pageant analogy, don’t forget the robots would be programmed by someone with their own subjectivity on what would warrant a 0.1 deduction as opposed to a 0.2 deduction. 

So effectively, 

- human judges are trying to be objective with their subjectivity

- robot judges are being objective but reliant on the subjectivity of the programmer (for point deductions).

= same diff!

Certainly, but humans err even on their own rules, as we have seen. 

At the very least an algorithm programmed with humans will be able to repeat the steps better than the human who programmed it. That's all I mean. Of course some data entry upfront has to happen, and bias will happen. But even that bias can be documented and even shared as it will be forever encapsulated within the program and thus could be open to scrutiny. 

Not quite same diff, thus. A human when presented with the same game in the same state years apart may grade them differently. A robot would not. 

[Link | 2,709]

Facial recognition is consumer-grade tech now, and works despite changes in facial hair, age, angle, lighting, and expression. I think software can be written and taught to rate physical condition of cardboard and shrinkwrap. 

[joshdose | 56]

I personally believe that the cost to properly design and implement a robotics system of this caliber would be more expensive than any toy grading company would ever be worth. So, it’s never gonna happen.

[Gloves | 11,771]

9 minutes ago, joshdose said:

I personally believe that the cost to properly design and implement a robotics system of this caliber would be more expensive than any toy grading company would ever be worth. So, it’s never gonna happen.

I dunno how much the companies are worth no can't speak to that, but I can say with relative confidence that setting up a machine learning ai is not particularly expensive. 

That said, I agree it's not likely to happen. 

I'm just saying that it would be better for the consumer if there was a more consistent system, and I'm proposing that an AI would be an appropriate solution. 

In much the same way that you can look at 60 graded games and get a sense from that experience when looking at a 61st game what it might grade at, a machine learning ai learns in exactly the same way. You'd feed it many photos of games, tell it what grade they are and why, and it'll learn and adjust and eventually be better than you at establishing grades. You almost literally just feed it many many photos, that's about it.