The Game Outcomes Project, Part 4: Crunch Makes Games Worse
遊戲專案為何成功系列之四：加班反而會把事情搞砸
網誌版：http://wp.me/pBAPd-qJ
原文網址：
http://gamasutra.com/blogs/PaulTozour/20150120/234443/The_Game_Outcomes_Project_Part_4_Crunch_Makes_Games_Worse.php
縮網址：http://tinyurl.com/m7kmuzf
撰文：Paul Tozour
繁體中文翻譯：NDark
20150120
譯按：本文是一篇統計學專業文章，若有翻譯不正確的文句，請以原文為主。
This article is the fourth in a 5-part series.
Part 1: The Best and the Rest is also available here: (Gamasutra)
(BlogSpot) (in Chinese)
Part 2: Building Effective Teams is available here: (Gamasutra)
(BlogSpot) (in Chinese)
Part 3: Game Development Factors is available here: (Gamasutra)
(BlogSpot) (in Chinese)
This article is Part 4, and a Chinese translation will soon be available.
Part 5 will be published in late January 2015.
For extended notes on our survey methodology, see our Methodology blog
page.
Our raw survey data (minus confidential info) is now available here if
you'd like to verify our results or perform your own analysis.
The Game Outcomes Project team includes Paul Tozour, David Wegbreit, Lucien
Parsons, Zhenghua “Z” Yang, NDark Teng, Eric Byron, Julianna Pillemer, Ben
Weber, and Karen Buro.
本文是五篇系列中的第四篇。
第五篇將會在一月底釋出。
想要知道問卷的方法論，請參閱部落格頁面
:http://intelligenceengine.blogspot.com/2014/11/game-outcomes-project-methodology-in.html
我們問卷的原始資料在此，有興趣的朋友可逕自取用分析。
"遊戲專案為何成功"團隊成員包含Paul Tozour，David Wegbreit，Lucien Parsons，
Zhenghua “Z” Yang，NDark Teng，Eric Byron，Julianna Pillemer，Ben Weber，及
Karen Buro。
The Game Outcomes Project, Part 4: Crunch Makes Games Worse
遊戲專案為何成功系列之四：加班反而會把事情搞砸
Extended overtime (“crunch”) is a deeply controversial topic in our
industry. Countless studios have undertaken crunch, sometimes extending to
mandatory 80-100 hour work weeks for years at a time. If you ask anyone in
the industry about crunch, you’re likely to hear opinions stated very
strongly and matter-of-factly based on that person’s individual experience.
And yet such opinions are almost invariably put forth with zero reference to
any actual data.
延長工時（加班）在我們的產業中充滿爭議。無數的工作室都曾採取加班的手段，甚至一
周會工作八十到一百小時。假如我們詢問業界加班的情形，我們會聽到各種基於個人經驗
的不同看法。
但這些意見純粹都是主觀意見，缺乏實際數據佐證。
If we truly want to analyze the impact of extended overtime in any scientific
and objective way, we should start by recognizing that any individual game
project must be considered meaningless by itself – it is a single data
point, or anecdotal evidence. We can learn absolutely nothing from whether a
single successful or unsuccessful game involved crunch or not, because we
cannot know how the project might have turned out if the opposite path had
been chosen – that is, if a project that crunched had not done so, or if a
project that did not employ crunch had decided to use it.
As the saying goes, you can’t prove (or disprove) a counterfactual – you’d
need a time machine to actually know how things would have turned out if you’
d chosen differently.
假如我們真的想要用科學化的方式分析加班帶來的衝擊，我們應該先認知道一點也就是：
分別從各專案的特例來看都是沒有意義的。我們無法清楚的知道加班對於專案是否有影響
，因為我們從事後來看只能看到成功與失敗，而不能用另一個方式再執行同一個實驗，因
為我們還沒發明時間機器。
Furthermore, there have undeniably been many successful and unsuccessful
games created both with and without crunch. So we can’t give crunch the
exclusive credit or blame for a particular outcome on a single project when
much of the credit or blame is clearly owed to other aspects of the game’s
development. To truly measure the effect of crunch, we would need to look at
a large sample, ideally involving hundreds of game projects.
更進一步，無法否認的有很多成功或失敗的專案都可能由加班或不加班的情形下完成。所
以我們不能完全用加班來指責單一專案的成敗，因為造成他們的成功與失敗可能來自不同
的要素。真正要測量加班的影響，我們應該用巨觀，數百個專案的數據來做。
Thankfully, the Game Outcomes Project survey has given us exactly that. In
previous articles, we discussed the origin of the Game Outcomes Project and
our preliminary findings, and our findings related to team effectiveness and
many additional factors we looked at specific to game development. We also
wrote up a separate blog post describing the technical details of our
methodology.
In this article, we present our findings on extended overtime based directly
on our survey data.
剛好"遊戲專案為何成功"的問卷給了我們這個機會。在先前的系列文章中，我們已經談論
了遊戲專案為何成功這個計劃的來由與初步結果，找到與團隊效率之間的關係，以及遊戲
製作領域的額外要素。我們也在部落格撰寫了我們的方法論。
本篇文章中，我們從問卷的數據中持續尋找加班相關的線索。
Attitudes Toward Crunch
Developers have surprisingly divergent attitudes toward the practice of
crunch. An interview on gamesindustry.biz quoted well-known industry figures
Warren Spector and Jason Rubin:
對於加班的不同態度
令人驚訝地，開發者對於加班的態度也很分歧。在Gamesindustry.biz的訪問中我們可以
引述Warren Spector 與 Jason Rubin 的說法：
“Crunch sucks, but if it is seen by the team members as a fair cost of
participating in an otherwise fantastic employment experience, if they value
ownership of the resulting creative success more than the hardship, if the
team feels like long hours of collaboration with close friends is ultimately
rewarding, and if they feel fairly compensated, then who are we to tell them
otherwise?" asked Rubin.
Rubin說："加班確實糟透了，但假如從團隊成員的角度看來，那可能也是一個美妙的團隊
經驗，假如他們認為創作的結果高過痛苦，假如他們認為與親密戰友長時間的合作是一個
終極的滿足，假如他們能夠獲得回饋，那麼誰又有資格跳出來阻止他們？"
[…] "Look, I'm sure there have been games made without crunch. I've never
worked on one or led one, but I'm sure examples exist. That tells me
something about myself and a lot about the business I'm in," said Spector.
Spector繼續說："...聽著，我確信一定有不需要加班就產出的遊戲，但我待過的開發案
從未這樣，雖然我相信一定有例外。這就是我所作的工作與產業。"
[…] "What I'm saying is that games - I'm talking about non-sequels,
non-imitative games - are inherently unknowable, unpredictable, unmanageable
things. A game development process with no crunch? I'm not sure that's
possible unless you're working on a rip-off of another game or a low-ambition
sequel.
"...我不是說那些續作專案，抄襲遊戲，而是說完全原創，完全未知的產物。這種遊戲開
發案子怎麼可能不加班？除非你正在抄襲或是只是作沒有野心的續作。"
“[…] Crunch is the result of working with a host of unknown factors in
creative mediums. Since game development is always full of unknowns, crunch
will always exist in studios that strive for quality […] After 30 years of
making games I'm still waiting to find the wizard who can avoid crunch
entirely without compromising at a level I'm unwilling to accept.”
"...加班是因為我們是與未知參數的藝術媒體戰鬥過程的產物。只要遊戲開發充滿了不確
定性，為了追求品質，加班就是必然。但說起來簡單，三十年的遊戲製作機驗後，我仍等
待某個魔術師來告訴我不需要加班就可以做出我可以接受的關卡。"
On the other side of the fence is Derek Paxton of Stardock, who said in an
interview with Gameranx:
在Gameranx的訪問中，Stardock 的 Derek Paxton 表達了另一個角度的看法：
“Crunch makes zero sense because it makes games worse. Companies crunch to
push through on a specific game, but the long-term effect is that talented
developers, artists, producers and designers burn out and leave the industry.
"加班一點也沒有意義，因為它只會把遊戲搞砸。公司會用加班來壓縮特定遊戲專案，但
長期來看會把有才能的開發者，美術人員，製作人，設計人榨乾，逼得他們不得不離開這
個產業。"
“Companies and individuals should stop wearing their time spent crunching as
a badge of honor. Crunch is a symptom of broken management and process.
Crunch is the sacrifice of your employees. I would ask them why crunch isn’t
an issue with other industries. Why isn’t crunch an issue at all game
studios?
"公司與開發者應該停止把[時間花在加班]當作榮譽的象徵。加班是崩壞管理與流程的病
徵。加班是員工的犧牲品。我問其他產業為何他們不需要加班？為什麼不是每個遊戲工作
室都需要加班？"
“Employees should see it as a failure. Gamers should be concerned about it,
because in the long term the hobby they love is losing talent because of it.
Companies should do everything in their power to improve their processes to
avoid these consequences.”
"員工應該把這件事視為失敗。遊戲開發者應該認真關心此事，因為長期來看，他們對遊
戲的愛會因此而遺失。公司應該要盡其可能改善流程來避免這些後果。"
So who is right – Spector and Rubin, or Paxton?
所以誰才是對的？Spector 及 Rubin，還是 Paxton？
[Full disclosure: team member Paul Tozour leads Mothership Entertainment,
whose flagship game is being published by Stardock.]
[搶先報：Paul Tozour 率領母艦娛樂這間公司，他們的主打遊戲會被 Stardock 發布。]
In the Game Outcomes Project survey, we provided 3 text boxes at the end that
respondents could use to tell us about their industry experiences. Where
they mention crunch, they invariably mention it as a net negative. One
respondent wrote:
在遊戲專案為何成功的問卷中，我們設計了三個開放欄位給回答者，讓他們告訴我們產業
的經驗。關於提到加班的部分，不約而同地都提出負面的說法。其中一個回應這樣寫著：
“The biggest issue we had was that the lead said ‘Overtime is part of game
development’ and never TRIED to improve. As sleep was lost, motivation
dropped and the staff lost hope ... everything fell apart. Hundred-hour
weeks for nine months, and I'm not exaggerating. Humans can't function under
these conditions ... If you want to mention my answer feel free. I'm sure
it'd be familiar to many devs.”
"我們最大的問題就是管理者說：[加班是遊戲開發的一個部分]，而從未試著改善，當睡
眠不足，熱情與希望也會隨之降低與剝離。我說真的，人類不能在九個月每周上百個小時
的加班這樣的環境下工作... 假如你們希望我老實講，我敢保證其他團隊狀況都相同。"
Another developer put it more bluntly:
另一個開發者說得更難聽：
“Schedule 40 hours a week and you get 38. Schedule 50 and you get 39 and
everyone hates work, life, and you. Schedule 60 and you get 32 and wives
start demanding you send out resumes. Schedule 80 and you’re [redacted] and
get sued, jackass.”
"一周四十個小時的工作，那麼工作效率差不多是三十八小時。如果排五十個小時，那麼
就會得到三十九小時外加痛恨工作，痛恨人生，及痛恨管理階級的員工。如果排了六十個
小時，那麼會得到三十二個小時的效率外加離職潮。排八十個小時的工作，那麼只會收到
存證信函。"
In this article, we will be getting a final word on the subject from the one
source that has yet to be interviewed: the data.
這篇文章中，我們會訪問我們的案例，也就是那些我們手中的數據資料，對這個題目給一
個總結。
The “Extraordinary Effort” Argument
"超凡努力（加班，代表著熱情）"理論
We’ll begin by formulating the “pro-crunch” side of the discourse into
testable hypotheses. Although no one directly claims that crunch is good per
se, and no one denies that it can have harmful effects, Spector and Rubin
clearly make the case in the article above that crunch is often (if not
usually, or even always) a necessary evil.
雖然沒人直接聲稱加班本身就是好事，也沒有人否認它有害，Spector 與 Rubin 清楚地
在前面的說法也證實通常（並非總是）加班是必要之惡。但我們先試著以"加班是好事"這
個論點來做個整理。
According to this line of thinking, ordinary development with ordinary
schedules cannot produce extraordinary results. We believe an accurate
characterization of this viewpoint from the gamesindustry.biz article quoted
above would be: “Extraordinary results require extraordinary effort, and
extraordinary effort demands long hours.”
也就是假設這樣的思路下去思考，正常工期的開發方式沒辦法製作傑出作品。也就是相信
Gamesindustry.biz的訪問中所提到論點："超凡的成果來自於超越極限的努力（超凡的努
力），而超越極限的努力需要長時間付出，也就是加班。"
This position (we’ll call it the “extraordinary effort argument”) leads
directly to two falsifiable hypotheses:
1. If the “extraordinary effort argument” is correct, there should be a
positive correlation between crunch and game outcomes, and higher levels of
crunch should show a measurable improvement in the outcomes of game projects.
2. If the “extraordinary effort argument” is correct, there should be
relatively few, if any, highly successful projects without crunch.
這個論點（我們姑且稱為超凡努力的論點）直接就會發現兩個反證：
# 假如超凡努力的論點是對的，那麼在我們的問卷中加班與遊戲產出分數上會有正相關
，越加班，就應該會產出優秀的作品。
# 假如超凡努力的論點是對的，那麼應該不可能發生沒加班卻高度成功的專案。
Luckily for us, we have data from hundreds of developers who took our survey
with no preconceptions as to what the study was designed to test, and which
we can use to verify both of these statements. We’ll agree to declare
victory for the pro-crunch side if EITHER of these hypotheses remains
standing after we put it in the ring with our data set.
很幸運地，我們從問卷中得到數百分開發者的回應，可以透過這些數據來證實這兩件事，
尤其是在我們設計之初並沒有故意去設計這樣的實驗。假如數據中告訴我們超凡努力論點
是對的，那麼我們也會宣告加班是勝利之道，並將其放在結論的皇冠上。
Crunching the Numbers
We’ll approach our analysis in several phases, carefully determining what
the data does and does not tell us.
加班數字
我們接著從數個步驟來分析，小心地看那些數據透露，或沒有透露的事。
Our 2014 survey asked the following five questions related to crunch, which
were randomly scattered throughout the survey:
#“I worked a lot of overtime or ‘crunched’ on this project.”
#“I often worked overtime because I was required or felt pressured to.”
#“Our team sometimes seemed to be stuck in a cycle of never-ending crunch
/ overtime work.”
#“If we worked overtime, I believe it was because studio leaders or
producers failed to scope the project properly (e.g. insufficient manpower,
deadlines that were too tight, over-promised features).”
#“If I worked overtime, it was only when I volunteered to do so.”
我們在2014年的問卷中問了以下關於加班的問題，在問卷中我們還把它們都隨機排列：
# 我在專案中超時工作。
# 因為感受到壓力，我常常超時工作。
# 我們的團隊常常感覺到受阻礙，並陷入無止盡的加班。
# 需要加班的原因是領導層與製作人在時程上搞砸了。（人力不足，估計期限過短，過
度承諾）
# 我加班是因為我自願加班。
Here’s how the answers to those questions correlate with our aggregate
project outcome score (described on our Methodology page). On the horizontal
axis, a score of -1.0 is “disagree completely” and a score of +1.0 is “
agree completely."
這裡是這些答案與總和專案產出分數的相關分數（方法論在我們的部落格已描述），水平
軸是從-1.0的完全不同意，到1.0的完全同意。
Figure 1. Correlation of each crunch-related question with that project’s
actual outcome (aggregate score). Each of the 5 questions is shown, as an
animated GIF with a 4-second delay. Only the horizontal axis changes.加班相關
的問題與總和產出分數的關聯性，每個問題以一個四秒的週期顯示出來
The correlations are as follows: -0.24, -0.30, -0.47, -0.36, +0.36 (in the
same order listed in the bullet-pointed list above). All five of these
correlations have statistical p-values well below 0.001, indicating that they
are statistically significant. Note how all the correlations are strongly
negative except for the final question, which asked whether crunch was solely
voluntary.
關聯性依序是-0.24，-0.30，-0.47，-0.36，0.36（順序如問題序）。五個關聯性都有少
於0.001的統計p值。也就是具有統計表徵。注意除了最後一個問題自願加班之外，這裡關
聯性都是強烈的負向。
“But wait,” a proponent of crunch might say. “Surely that’s only because
you’re using a combined score. That score combines the values of questions
like ‘this project met its internal goals,’ which are going to give you
lower values, because they're subjective fluff. Of course people who are
unhappy about crunch are going to give that factor low scores – and that’s
going to lower the combined score a lot. It’s a fudge factor, and it’s
skewing your results. Throw it out! You should throw away the critical
success, delays, and internal goals outcomes and JUST look at return on
investment and I bet you’ll see a totally different picture.”
但加班的支持者可能會說："等等，這一定是因為這裡是一個總合分數，包含了內部滿意
度，這當然會有負分，因為那是主觀意見，加班就是會讓人不開心，才會導致總合分數這
樣發展，應該要排除在外！我們應該要只看利潤的產出分數，一定可以看到不同的結果。
"
OK, let’s do that:
那麼我們也從善如流：
Figure 2. Correlation of each of the 5 crunch-related questions with that
project’s return on investment (ROI). As with Figure 1, each of the 5
questions is shown, as an animated GIF with a 4-second delay. Only the
horizontal axis changes. Note that many of the points shown represent
multiple coincident points. See our Methodology page for an explanation of
the vertical axis scale.五個加班相關問題對上專案利潤的關聯度，如圖一相同，每個
問題以一個四秒的周期顯示。只有在水平軸不同。每個點都可能代表重合在一起的點。在
垂直軸的縮放方式請參照我們的部落格網頁。
Notice how the lines have essentially the same slopes as in the previous
figure. The correlations with ROI are as follows (in the same order): -0.18,
-0.26, -0.34, -0.23, and +0.28. All of these correlations have p-values
below 0.012.
注意到迴歸線仍與前一張圖相同嗎？利潤的關聯性是：-0.18，-0.25，-0.34，-0.23，及
0.28。全部關聯性都有小於0.012的統計p值。
Still not convinced? Here are the same graphs again, correlated against
aggregate reviews / MetaCritic scores.
不相信嗎？這張圖也一樣，對上網頁分數的關聯性：
Figure 3. Correlation of each of the 5 crunch-related questions with the
project’s aggregate reviews / MetaCritic score (note that the vertical axis
does not represent actual MetaCritic scores but is a normalized
representation of the answers to this question; see our Methodology page for
more info). As with Figures 1 and 2, each of the 5 questions is shown, as an
animated GIF with a 4-second delay. Note that many of the points shown
represent multiple coincident points. Only the horizontal axis changes.五個加
班問題對上網頁分數的關聯性（注意垂直軸並非表示MetaCritic真正分數，而只是一個對
問題經過正規化的數值。更多資訊，請看我們的部落格）如同圖一與圖二，五個答案都以
一個四秒週期的方式顯示。每個點都可能代表重合在一起的點。只有在水平軸的參數是不
同的。
The results are essentially identical, and all have p-values under 0.05.
結果一樣，全部都具有小於0.05的統計p值。
So if our combined score has a negative correlation with ALL our crunch
questions except the one about crunch being purely voluntary (which itself
does not imply any particular level of crunch), that means that we’ve
disproven the first part of the “extraordinary effort argument” – the
correlation is clearly negative, not positive.
總合的分數對上除了自願加班之外的所有加班問題都是負向的關聯。意思是我們能夠推翻
超凡努力論點，很清楚，就是沒有正相關。
Now let’s look at the second testable hypothesis of the “extraordinary
effort argument.”
In Figure 4 (below), we’re looking at the two most relevant questions
related to overall crunch for a project. The vertical axis is the aggregate
outcome score, while the horizontal axis represents the scale from “disagree
completely” (-1) to “agree completely.” The black lines are trend lines.
As you can see, in both cases, higher agreement with each statement
corresponds to inferior project outcomes.
接著來看看我們對於超凡努力理論的第二個辯證。
在下面的圖四中，我們取出兩個對加班問題中最相關的問題。垂直軸是總合產出分數，同
時水平軸是從完全不同意的-1，到完全同意。黑色的線是趨勢線。如你可見，在兩個問題
中，越高的同意帶來越低的總和分數。
Figure 4. The two most relevant questions related to crunch compared to the
aggregate project outcome score.兩個相關問題對上總和的產出分數。
We’ve added horizontal blue and orange lines to both images. The blue line
represents a score of 80, which will be our subjective threshold for “very
successful” projects. The orange line represents a score of 40, which will
be our threshold for “very unsuccessful” projects.
我們接著加上了的藍色與橘色水平線。藍色線是80，也就是我們主觀認定非常成功專案。
橘色線則代表40。也就是我們主觀認定非常不成功的專案。
The dots above the blue line tell a clear story: in each case, there were
more successful games made without crunch than with crunch.
在藍線之上的點的分布清楚了代表一件事：多數成功的遊戲沒加班的數量比加班的多。
However, these charts don’t tell the full story by themselves; many of the
data points are clustered at the exact same spot, meaning that each dot can
actually represent several data points. So a statistical deep-dive is
necessary. We’re particularly interested the four corners of the chart –
the data points above the blue line on the extreme left and right sides of
each chart (below -0.6 and above +0.6 on the horizontal axis) and below the
orange line on the left and right sides.
然而，只是圖並不能將細節全盤托出，很多數據點都重合在一起，分不清楚有幾個。所以
我們需要再深一步的統計分析。我們對這張圖的四個角落特別有興趣。也就是藍線以上的
左右端，以及橘線以下的左右端。（水平軸以-0.6及0.6為界線）
Looking solely at the chart on the top of Figure 4 (“I worked a lot of
overtime or ‘crunched’ on this project”), we observed the following
pattern. Note that the percentages are given in terms of the total data
points in each vertical grouping (under -0.6 or above 0.6 on the horizontal
axis).
獨立看圖四上半（我在專案中超時工作。）我們觀察到後述的模式。注意那些比例是以水
平軸已經切開（為左右兩群組）的群組來計算。
We can see clearly that a higher percentage of no-crunch projects succeed
than fail (17% vs 10%) and a much larger percentage of high-crunch projects
fail rather than succeeding (32% vs 13%). Additionally, a higher percentage
of the successful projects are no-crunch than high-crunch (17% vs 13%), while
a higher percentage of the unsuccessful projects are high-crunch vs no-crunch
(32% vs 10%).
我們可以很清楚地看到非加班的區塊成功數量是高於失敗數量（17%多過於10%），高度加
班的區塊中，失敗卻高於成功（32%多過於13%）。成功專案中，不加班多於加班（17%多
過於13%）。失敗專案中高度加班的情形多過於不加班（32%多過於10%）。
Here’s the same chart, but this time looking at the bottom question, “Our
team sometimes seemed to be stuck in a cycle of never-ending crunch /
overtime work.”
同樣的圖表中，我們看下半個問題：我們的團隊常常感覺到受阻礙，並陷入無止盡的加班
。
These results are even more remarkable. The respondents that answered “
disagree strongly” or “disagree completely” were 2.5 times more likely to
be working on very successful projects (23% vs 9%), while the respondents who
answered “agree strongly” or “agree completely” were, incredibly, more
than 10 times more likely to be on unsuccessful projects than successful ones
(41% vs 4%).
結果更強烈。在不加班（回應強烈不同意與完全不同意）的案子中成功專案是超過不成功
專案的兩倍半（23%對上9%）。在高度加班中（強烈同意與完全同意）的案子中不成功的
案子則是成功案子的四倍（41%對上4%）
Some might object to this way of measuring the responses, as it is an
aggregate outcome score which takes internal achievement of the project goals
into account – and this is a somewhat subjective measure. What if we looked
at return on investment (ROI) alone? Surely that would paint a different
picture.
有些人可能會質疑總和的分數包含了專案的內部滿意度，當然就是主觀意見。那假設我們
只看專案利潤？會有不同的結果嗎？
Here is ROI:
專案利潤的圖在此：
Figure 5. The two most relevant questions related to crunch compared to
return on investment (ROI).最相關的兩個問題對上專案利潤的關聯性
The first question (top chart) gives us the following results:
第一個問題的結果如此：
The second question (bottom chart) gives us:
第二個問題的結果如此：
These results are essentially equivalent to what we got with Figure 4