Bug Bug: (???) Bad Settings (???)

Sort:
cwfrank

@chuckmoulton wrote ...

"Still very buggy... apparently my partner and I had 3 games open at the same time. I didn't see any of them. By the time my partner told me there were games, I had lost 1 to game abandoned (-12 points), won 1 to resignation (+0 points), and was 15 seconds down a third game that we eventually won (+0 points). Over 50% of my total losses on this server have been to server bugs with games not starting and losing without having a chance to make a move."

 

@cwfrank (me, myself and I) response ...

"Yo, @chuckmoulton -- references? (Forum post.) -- I've suspected something funky like what you describe for a while, given how often games are immediately or arbitrarily resigned or abandoned or timed-out. It would be helpful if you track and log specific circumstances."

 

=====

 

Point being ...

 

Sometimes I'm left searching, or immediately connected to a game ... and almost immediately I get a "resigned" or "abandoned" notice. (Sometimes with points assigned, other times just arbitrary.)

 

The whole "I WANT MY POINTS" factor was something I previously complained about with the same team or individuals regularly resigning or aborting games ... this shortly after initial v3 release / publication. That's "handled" in the sense that sometimes points are assigned, and other times the game is simply terminated. (Assuming some aggregated data measure to determine such factors.)

 

It is still of concern that the same set of team and/or random member can be assigned to the same game(s) repeatedly, without play, and the same team (or game members) can resign without penalty multiple times.

 

That said ...

 

@chuckmoulton's observation is relevant to the above articulation of details, such that settings may allow multiple games, but the UI may not be switching between or allowing or properly pushing multiple games to the client UI, and then negatively impacting players who are capable of such; or whose UI may be configured to allow such (client-server communication) and then not communicated back and forth properly.

 

Note: I have my settings such that multiple games are not allowed. Such that... if I have multiple seeks, all others are canceled when a game starts.

 

I'm irritated by connecting to a game, and then having it almost immediately terminated. (sometimes several times in a row, often by the same team or set of assigned players).

 

Note / Aside: I understand that such observations and demands make things significantly more computationally complex; but, since I don't have access to information to help this process along, I can't make any better or further suggestion beyond calling-out the observation. I'd love to help, but, I don't have data to be of any further assistance.)

 

And @chuckmoulton's observation is not irrelevant to this factor.

 

Blah

 

 

In fact, if all that are available are the same team and/or random members, and the same team or members decline to play against another team or randomly matched opponent(s) on a regular basis (repeated) ... there needs to be a more significant and regular penalty factor (in addition to @chuckmoulton's observation -- though it might cause @chuckmoulton some heartache, I think these two situations and circumstances are intertwined in a way that needs to be fixed or solved with some [deterministic] degree of finality, including public dissemination of the particular circumstances of the conditions that lead to one circumstance or situation, or the other).

 

$0.02 (paid in full)

 

Can we please get a response to how matching is done, and how and when multiple games are allowed, and how that impacts score ... especially when there are multiple bad (teams) or resigned or declined games in close sequence among the same set of players abandoning or resigning games. Oh, and, don't forget @chuckmoulton's observation(s).

 

Thanks, and sorry for the obtuse font and call-out.

chuckmoulton

Early on it was the case that when you clicked "rematch" and "new game" there was the possibility 2 games could start.  Then they fixed that and the bug didn't come up for weeks.  Now it seems to be back again.  This is pretty annoying because for those of us high rated we are delighted to ever see a match +1 or more.  I always offer rematch for those types of matches.  My opponents usually decline, then I click new game.  But sometimes I think they declined, click new game, then they accept later somehow.  Those are the situations when I get 2 games.

 

The other problem is sometimes a game will start and it will not come up for me or my partner at all.  I haven't been able to pin down any rhyme or reason to that.  I'm in Skype with my partner, so whoever actually sees the game will yell "MOOOOOOVE!!!".  Then the person who can't see a board will yell "There is a game?!!  Oh no..." and frantically try to refresh the browser.  Usually refreshing the browser brings up the game that wasn't showing (in the early days even that didn't work... you needed to login and logout).  Unfortunately sometimes by the time you figure out there is a game and click refresh, you're already through the 20 seconds and the game is marked "abandoned".

 

With abandoned games, points only accrue to the abandoned board, not the other (to both the player who didn't move and to his opponent).  This doesn't happen nearly as often as it did when chess.com first got bughouse... back then it was around 1/10 games.  Now it's more like between 1/100 and 1/200 games.  For high rated players who have to play 90%+ of games for +0, you've never supposed to lose those matchups and losing to a server bug abandoned means -16 points.  Even when you're able to refresh in time, you're down 20 seconds, which can be a decisive disadvantage when you're playing against a decent team.

erik

We have a fix coming for this very soon! Thanks for the report. You will now only be able to have one bughouse game at once. 

cwfrank
erik wrote:

We have a fix coming for this very soon! Thanks for the report. You will now only be able to have one bughouse game at once. 

 

Hi Eirk,

 

Thanks for the ACK (acknowledge).

 

Here's a perfect example ...

 

Game 1: https://www.chess.com/live/game/1869435115

Team Member: https://www.chess.com/live/game/1869435116

 

Game 2: https://www.chess.com/live/game/1869435374

Team Member: https://www.chess.com/live/game/1869435375

 

The first game, someone declines to play ... no big deal.

 

On the other hand, the second game ... same person paired with or against ... resigned twice in a row. (Refusal to play, bad sportsmanship.) -- There should be a stricter penalty. A resignation is a resignation. In any other form (or forum) a resignation would equate to the winner receiving points. Since this is Bughouse, and it's a team effort -- it's a little different. But, when someone refuses to play multiple times, there should be both a points penalty (1.5 or 2x), and a seek-time penalty against those who are unwilling to play.

 

Both highly rated and lesser rated players face the same or similar seek considerations, and it becomes much more difficult to find games if we limit on the simplified ratings seek settings we have now (against needing to find 4 players on random, or 3 players = 1 team + 2 random, or 2 teams as 2 players).

 

I understand that an improvement is forthcoming ... just wanted to articulate a specific example of what is an unacceptable and is happening too often. (It's an explicit action.)

 

Explicit vs. Implicit: "Server did not receive your move in time," or "Could not connect to and initialize game." -- These provide explicit information to the user. (Okay, no big deal.) If a resignation is "explicit" (button clicked, instead of implicit: "users not connected to game) ... points should be assigned, and the user resigning without moving in a team-oriented game should receive a double-points-penalty for refusal to play (especially if resigning or aborting games often). That would be fair.

 

I would even beg to argue that an individual who dumps or drops too many games by resignation or aborting (refusal) should simply face a 24-hour ban. "You've been banned from live chess for the next 24 hours for refusal to play. (Unsportsperson-like conduct.)" -- And while I know many people will argue that these measures are somewhat extreme in proposition ... they won't play if they don't like the terms or can't play on their own ambiguous terms. (Less clutter for the rest of us if those who refuse to play can't be as picky or choosy about playing on their own terms and have to abide by rules or face strict punishment.)

 

 

chuckmoulton

Since @erik seems to be reading this, I'll highlight another annoying bug -- even though it is unrelated to the thread subject.

 

When you follow a person playing bughouse, it displays both the game of who you follow and his partner.  However, the partner's board is displayed wrong half the time.  The partner's board is always displayed with white at the bottom instead of the teammate of who you are following at the bottom.  This means if the person you follow happens to be white, you will see 2 white players at the bottom, so his teammate is diagonal across instead of next to him.  This makes it very hard to understand what is going on in the game.  You can manually flip the board, which will only flip the main board, not the partner board.  This will put the person you are following at the top of the board with his partner next to him (if he is white and his partner is black).  The problem I mention only started about a week ago.

 

What should happen is the person you are following should be at the bottom of the board (white or black) and his partner should ALSO be at the bottom of the board (white or black) so that partners are next to each other when you observe or follow.  Flipping should flip BOTH boards so that BOTH players are now at the top of the board.  Or alternatively there could be separate flip controls for each board.

 

This is really basic stuff.  The fact that such an elementary error could be introduced is an indication that:

1) the programmers don't understand how bughouse works AND/OR

2) no one is bothering to test things at all (such as following a bughouse game)

 

I don't think this error is widely reported because an order of magnitude more people play bughouse than observe bughouse.

cwfrank

I've seen your suggestion / comment / feedback to this effect before, and, you're not the only one. Other people have hinted and suggested at the need for better observation.

 

Or, more pointedly ... "observation" and "review" kind of go hand-in-hand in terms of board and representational semantics. And people are begging for a better (game) review interface for both Bug and ZH. Watching (observing) a game should follow the same analysis board type of semantics as game review.

 

So ... the suggestion is out there, and probably actively WIP (work-in-progress), like many other things, just not yet implemented.

chuckmoulton

@cwfrank The difference is what you are mentioning (better examination) would be a new feature, whereas what I mentioned (orienting the boards correctly) is something that was correct from day 1 and for months, only to be screwed up 1 week ago.  Again, I too would like new features and improvements, but step 1 is don't make it worse.  As doctors say: first, do no harm.  Programmers should not be actively introducing bugs that weren't there before.

cwfrank

@chuckmoulton ... I did not realize (forgive me). I don't analyze or observe or watch that much except between tournament matches.

 

If it was working, and then it broke ... then ... I have to agree with you ... such is what I know from a software development perspective: Don't Fvck Things Up. Especially things that are already working.

 

Unfortunately, most software developers and business monkeys disagree with me on this point, and I'm somewhat unpopular in calling it out. (Humor me: Best of intent ... not always everyone's intent despite attempting to put on good impressions or fall silent when they fail... their own failings compounded many times over due to basic lack of comprehension or skills. -- Nothing really you can do about it other than to complain, or at least try to say something and be helpful, and then then laugh at the obviousness of any situation which presents itself.)

 

 

Note / Edit:

There's another side to this I didn't think about before ... (implicit circumstance) ... if something was broken, there's nothing wrong with breaking it further until a fix can be implemented. From my perspective, as a developer. Especially if there's a potential security issue. Technically speaking, I don't have a problem with that. For example: Before Creating this topic, which contains "Bad Settings" in the title ... I went and looked at Live Chess settings (what's changed since the last I looked at it shortly after the v3 and variants deploy). I could see an issue or circumstance where the option to "highlight" valid moves in newer variants might cause issues with the overall "board" class or method or function; be it play or review/analysis. (I have this option turned off since I'm an experienced chess player and I don't think experienced players should be using it. But this is just a personal thought and preference; optional ethics. So, I don't know what it looks like with the drop-chess variants, though I should probably check it out; without using it as a suggestive crutch.) Therein, if there's an issue, especially given drop-variants such as Bug or ZH, I might not care (as a developer) about (temporarily) screwing things up unless or until I can implement a stable, long-term permanent fix (and, that may be work-in-progress).

 

 

 

cwfrank

Here's another example ...

 

Game 1: https://www.chess.com/live/game/1869540393

Team Member: https://www.chess.com/live/game/1869540392

 

Game 2: https://www.chess.com/live/game/1869540212

Team Member: https://www.chess.com/live/game/1869540211

 

I like playing with and against @ficslagger, but, despite liking this user ... this whole dumping games thing is purely unacceptable. @ficslagger should be punished or penalized for dumping games.

 

This ... If it's not a server and client-UI communications issue. IF it is a server and client-UI communications issue, then we need one of those flags like: "Your move did not reach the server in time," instead of seeing an arbitrary and repeating resignation in our stack of games.

 

If the resignation is explicit, then, it needs to be (progressively) penalized if continually repeated by the same user (such as abandoning or resigning too many games; the usual "fair play" policy or violation, etc).

 

One instance between the last observation (not this post, the previous), and this observation did not show in my games list ... which leads me to believe that the problems are in part systemic (as in a problem with the system, such that I can't provide explicit documentation of the issue, because it resides with other client-UI-to-server communications).

 

On the other hand, here's another example ...

 

Finished Game: https://www.chess.com/live/game/1869646862

Team Member: https://www.chess.com/live/game/1869646861

 

I was playing with @BiggerBishop (who, by the way, has an cool and interesting username and profile picture and profile description -- kinda funny-cool in the adult sense of things). We lost that game. Earlier in the evening (before this game), the two of us had a little spat as team members. (Whatever.) The next two games, however ...

 

Game 1: https://www.chess.com/live/game/1869650855

Team Member: https://www.chess.com/live/game/1869650856

Game 2: https://www.chess.com/live/game/1869651085

Team Member: https://www.chess.com/live/game/1869651086

 

@BiggerBishop dumped two games in a row.

 

If we're taking points away from individuals who do not want to play, this disenfranchises (unfair toward) @BiggerBishop's team member. In both the willingness to play department, and in the points department.

 

Thinking about that, logically, if I were to be writing the software to drive this mechanism ... if someone dumps two games in a row ... they have to wait 2-minutes to be paired (randomly) with someone again. The same holds for a coupled team. (So, if one or both or either member of a team starts arbitrarily or alternately resigning or dumping games, they've gotta wait 2+ minutes before starting another.)

 

Dumping one game is one thing. (It could happen on accident. Hold for a second, make sure you're not going to click wrong again. Okay? Got it? Good. Next game.)

 

Dumping a second game, for whatever reason -- this is poor form, bad sportsmanship (or bad sports-person conduct).

 

And a penalty should be leveled against a person or team in a relative sense. (Hopefully without the negative impact taking effect on a randomly assigned team member who is unaware that someone they're randomly assigned with has a [historical] bad attitude toward playing a game.)

 

People will obviously object to this. But, if they do, ask them to break-down the logic the way they see things.

 

Here's my personal statement: If I dump two games, and I know the rules ... well ... damnit, I violated the rules so I've gotta eat the cost of a penalty. (Hopefully this doesn't negatively impact my team member, but, if it does, I'll take the complaint and the tongue-lashing, etc... because I deserve the fruits of my own bad behavior, etc...) The same as we know about abandoning or timing-out of games too much or too often, etc...

 

 

cwfrank

Better Yet:

 

If a randomly assigned player (not paired-team) dumps two games in a row, as assigned by the server ...

 

Don't take points away from the unwitting participants (team members who do not know their team members are sand-bagging them)...

 

... put-up a notification, like the "1-point bye" in tournaments, such that a user is informed additional points have been deducted from their rating, and assigned to both of the opposing team for failure to play (and, you've got a 2-minute time-out unless or until you can behave appropriately).

 

I think that would kick some people's bad attitudes into gear.

 

Obviously this has to be weighed against the loss of players because too many people will probably complain about this because they don't see the harm or foul in starting and then abandoning or resigning or timing-out games. (Though, temporary until people get used to the fact they've gotta play or get lost and get gone for refusal to conduct themselves appropriately.)

 

Again, an obtuse thought (that many people will dislike) ... but it will get rid of the riff-raff and bad-eggs who only want to play on their own terms instead of fair play (within what the system can reasonably handle in relation to demands, expectations and requirements).

 

$0.02

 

cwfrank

The again (just talking to myself now) ...

 

... who am I kidding about prompting people about being penalized ...

 

... if people want to penalize themselves by having to seek longer for games ... let that be on them ... and, without prompt ... just increase the seek-time between games when they start dumping muliple games.

 

And, if they contact support about not being able to connect to games ... support can respond with: "You've dumped X number of games, and your Y wait-time is: Z-seconds (or minutes)"

 

Play games or get gone.

 

That's what it amounts to, right?

chuckmoulton

I never resign games myself.  I play anyone.

 

However, many people do resign games on move 1 often.

 

The reason is:

1) there is no penalty AND

2) there are annoying matchups they don't want

 

You seem to be focusing on #1.  The real problem is #2.  On most other servers it is possible to ABORT a game on move 1.  Chess.com treats a RESIGN on move 1 like an ABORT, so people resign when they want to abort.

 

On most other servers the matchups are better because you can control them by matching who you want or (if random) you are matched against someone of similar skill (with random pools in chess and other variants).  Chess.com currently makes it very difficult to specify a rating range for seeks.  This can be done within 400 point ranges, but only works for 1 partner when it works at all -- which it usually doesn't.  Users ought to be able to specify an exact number rather than limiting options to -400 to +400 by 100 point jumps.

 

Chess.com makes it impossible to specify a rating range for your partner.  High rated players are often stuck with played rated less than 1000 unless they manually partner.  This is completely unacceptable for anyone who doesn't want to tear all his hair out every game, as super-low-rated partners invariably never listen, hang all their pieces, and walk into mate even up a minute.

 

Finally, chess.com games are rated.  With a very bad matchup, you are playing for +0 -16.  There is no benefit to winning and some small chance you may lose -- when that happens you lose a lot of points.  By hitting RESIGN you effectively ABORT, avoiding the risk of losing 16 points for the benefit of gaining 0 points.  Some people who are very risk averse naturally choose to avoid +0 matches and only play +1 or more.  +1 or more also tend to be more competitive, interesting games.

 

In my opinion, the following things should happen (#3 & #4 are outside the scope of this post):
1) people should be able to ABORT games on move 1

2) people should be able to ABORT games by agreement in the middle of the game

3) people should be able to DRAW games after a 3-fold repetition

4) people should be able to DRAW games by agreement

5) people should be able to specify a rating range for OPPONENT seeks

6) people should be able to specify a rating range for PARTNER seeks

7) people should be able to optionally require +1 or more for games

 

For my part, I'll continue to play even with these deficiencies.  I take all comers for matchups and play for +0 over 90% of the time.  I only partner my friends manually, so I don't suffer from the lack of partnership seek settings.  I block anyone who abuses chess.com's bad draw rule by repeating a position 100+ times.

erik

Thanks! I will review. 

cwfrank
chuckmoulton wrote:
The reason is:

1) there is no penalty AND

2) there are annoying matchups they don't want

 

You seem to be focusing on #1.  The real problem is #2.  On most other servers it is possible to ABORT a game on move 1.  Chess.com treats a RESIGN on move 1 like an ABORT, so people resign when they want to abort.

 

...

 

I agree with all of your observations.

erik

thanks guys. we are going to improve the matching and the aborting. the hard part is that there are two schools of thought: 

#1 "I want closer matchups so I should be able to abort games"

vs

#2 "I want everyone to play any matchup and they should be punished for aborting"

They are opposites, so it's kind of a dance...

chesskingdreamer

Hard to see why anyone would advocate for #2.

cwfrank
erik wrote:

thanks guys. we are going to improve the matching and the aborting. the hard part is that there are two schools of thought: 

#1 "I want closer matchups so I should be able to abort games"

vs

#2 "I want everyone to play any matchup and they should be punished for aborting"

They are opposites, so it's kind of a dance...

 

chesskingdreamer wrote:

Hard to see why anyone would advocate for #2.

 

Just to clarify something, I'm not advocating for #2 in the form it's mentioned ... "I want everone to play any matchup and they should be punished for aborting." -- That would be obtuse. And that's not what I'm getting at.

 

On the other hand, Chess.com already has a warning about the right to "abort" being taken away if someone times-out or aborts games too often.

 

Let me rephrase: Chess.com has set that historical rule and precedent. Turning around and allowing one variant to operate different from another due to not implementing better ratings seek controls ... that's not an end-user problem, but an issue that many of us noticed upon initial deploy.

 

A "resign" is a resign. (The only other option if you time-out too much and have the right to abort taken away.) After a point, the same should hold for arbitrarily letting games time-out instead of hit "resign." -- Someone does this several times in a row, penalties need to take effect. I've provided examples where there are (2) dumped games. I've got examples where there are (3) and (4) dumped games.

 

That's what the server assigned. Play or get gone if it's the best match the server can make at the moment. By the time that poor match-up is concluded, the queue has probably refreshed such that there will be additional possibilities and options. (Though, sometimes the same people ending a game at the same time are put right back in queue in approximately the same location, such that the same or a similar match will come up randomly.)

 

I turned off my ratings seek settings to play Bughouse, upon initial deploy, to get games. But this is not representative of other variants.

 

We need to be able to set ratings seek settings BY-VARIANT. And then for specialized variants, like Bughouse, possibly a secondary branch of who to pair with (either ratings, and/or preference for friends).

 

=====

 

A while ago @chuckmoulton said this ...

"If I can't play bughouse with my friends, I'm not going to play on chess.com. It's that simple. There are a lot of people who play for fun with their friends. If you want to reduce the player pool dramatically, then force people to partner people who don't know how to play rather than allowing them to play with their friends."

 

And @chuckmoulton is right ...

The ability to team with a friend, or another member or player you play well with is one of those nice features. And that should not be taken away. (For whatever concern or reason this was said.)

 

Instead, it should be expanded to allow teams to seek other teams in queue.

 

=====

 

Therein, I have an obtuse thought or suggestion:

 

An altogether separate live chess server and user interface specifically for Bughouse.

 

(Obtuse) Because this is a lot of time and money.

 

I think some of the problems with Bughouse is (or are) that it is constrained by the existing, historical UI and settings. ZH fits nicely into the existing 1:1 UI. Bughouse is a different beast. We can't play Bughouse on one tiny little coffee-house table IRL. We've gotta grab 2 tables and 4 chairs, with enough elbow room to allow for movement and throw pieces around (the sides of the boards).

 

Teams should be able to find and challenge other teams. Right now, I'm not familiar with the ability to do this. Providing a way to do this would add more clutter to the existing live-chess (client) UI. A team gets together, and, it would be nice to see a team queue, and to specify things like only playing other teams, or allowing a team to be randomly matched (within some ratings-seek constraints).

 

Plus, it would be nice to see our team member's board side-by-side (same size), and set board arrangement preferences, etc... (We need better analysis, observation and review boards for Bug and ZH as well.)

 

Opposed to obtuse thoughts, better ratings-seek settings for Bug, and by-variant ratings seek settings are a practical must.

 

=====

 

Outside of any other chiming in with "agree" or "disagree" thoughts or observations I see or read ... I've already said too much on this topic. So, I'm going to stop here unless I see another alternative suggestion I would throw my support behind. (And try to keep it simple.)

 

But please don't peg me in the hole of: "Play match-up because I say so." -- Nay, this is a historical precedent that Chess.com has set, and people are "gaming" the system right now ... sometimes resigning, and at other times letting games time-out ... and facing no penalty (or too little) for violating the rules of an established system.

 

On the other hand ... fine ... bad matches because of server constraints while "work-in-progress," then change the rules of the system so that none of Chess.com's players in whatever variant are negatively affected by rules its established -- let anyone abort or abandon any game before play, without warning about having the 'abort' function or feature taken away (and the implicit have to resign or timeout or face a fair play warning). If Bug is that much different that it has different server rules and settings, etc... then... does it not deserve it's own server and UI given it's a Team Sport, and not 1v1 play?

 

$0.02 -- Best I got.

Sorry for being an obtuse thorn.

cwfrank

@chuckmoulton says ...

"It is RIDICULOUS how long it takes to get a game here. Average is 10 minutes between each game."

 

Tenatively I have to agree.

 

The recent test or "evolution" of seeing "Seeking rating match between ABC and XYZ..." (while seeking on random, or even teams seeking other teams, random or matched) ... this is not working. We need another matching tactic.

 

If the current iteration is just a stint (and/or test) ... (can we get an estimate of or about) how long until we're able to set these specific parameters in our settings? Such that, point being ... people only have complain about limitations they implement themselves.

 

In truth ... @chuckmoulton's comments (and my own observations) are meant to help refine things. But, if they're hurting more than helping, then implementation ideas (being tested) ... they need to be rolled back and another idea tried and tested.

 

Can we get communication (information) about the status of such parameters, ideas, tests, etc...

 

I don't mind being a guinea pig, but, I'd like to be in-the-know about what's going on behind the scenes (a personal statement).

chuckmoulton

Well, not only does it take forever to get a game, but also they clearly haven't implemented anything that has been discussed.  I still get +0 games after waiting 20 minutes.

 

It ought to be an opt-in setting: only play when between +X and -X win/loss of points.  That way even if I have that set, if it is taking a long time to get a game I could uncheck it if I wanted and get any opps.

cwfrank
chuckmoulton wrote:

Well, not only does it take forever to get a game, but also they clearly haven't implemented anything that has been discussed.  I still get +0 games after waiting 20 minutes.

 

It ought to be an opt-in setting: only play when between +X and -X win/loss of points.  That way even if I have that set, if it is taking a long time to get a game I could uncheck it if I wanted and get any opps.

 

Dude, seriously, calm down a little. It takes time. They're operating on a traditional 1:1 (or, 1v1) interface ... and they've got to update the code. (Go pull the code our web-based clients pull down via browser, format, and look at it for yourself.)

 

You can't turn this on a dime. They're working hard, and they're working fast. But if you push someone to work too hard, too fast, you could either lose the person who knows the code the best, or burn them out. We provide suggestions ... and modifying the code to work from a traditional and mostly 1:1 (or, 1v1) interface to integrating a new 2:2 (or 2v2) interface probably takes some time ... and then you implement settings.

 

They're working on it, actively ... they've already implemented and are obviously trying settings ... and it takes time to correlate and gather data ... and we've reported back our observations about changes we do know about.

 

Give them a chance to do something with this information ... can't be turned on-a-dime just because we don't like something. And it's not always as simple as a roll-back when you have certain code and ideas you want or need to preserve for future iterations, etc...