HCAL 2017 Phase1 Workflows #15889

kpedro88 · 2016-09-17T06:17:37Z

Long-anticipated... thanks to @mmusich for the autocond in #15884.

This PR adds two new workflows: 2017HCALdev ("old" pixel with new HCAL geometry/conditions/customizations for 2017) and 2017AllNew ("new" pixel with new HCAL).

Tested using workflows 10624.0, 10824.0 (ttbar, HCALdev and AllNew respectively).

@igv4321, @mariadalfonso, let me know if I missed any reco settings you want to have turned on by default.

attn: @abdoulline, @lihux25, @hatakeyamak, @pdudero

kpedro88 · 2016-09-17T06:18:01Z

@cmsbuild please test

cmsbuild · 2016-09-17T06:18:01Z

A new Pull Request was created by @kpedro88 (Kevin Pedro) for CMSSW_8_1_X.

It involves the following packages:

Configuration/Eras
Configuration/PyReleaseValidation
Configuration/StandardSequences
DQM/HcalTasks
EventFilter/HcalRawToDigi
EventFilter/RawDataCollector
RecoLocalCalo/Configuration
RecoLocalCalo/HcalRecProducers

@dmitrijus, @cvuosalo, @emeschi, @franzoni, @fabozzi, @cmsbuild, @srimanob, @slava77, @mommsen, @vanbesien, @hengne, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @makortel, @GiacomoSguazzoni, @rovere, @VinInn, @Martin-Grunewald, @deguio, @mariadalfonso, @dgulhan this is something you requested to watch as well.
@slava77, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

cmsbuild · 2016-09-17T06:18:16Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/15233/console

cmsbuild · 2016-09-17T08:51:02Z

-1

Tested at: a19eefb

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
813e5ad
51dd54a
12077f8
851461b
5cb0fc2
43dfd1b
032ad55
3cfd184
4c378a3
f569019
941fa6e
a53847c
a03ca88
bc0c679
af763d9
ec5ff3c
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15889/15233/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15889/15233/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15889/15233/summary.html

I found follow errors while testing this PR

Failed tests: RelVals

RelVals:

When I ran the RelVals I found an error in the following worklfows:
20824.0 step3

runTheMatrix-results/20824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2023D3_GenSimFull+DigiFull_2023D3+RecoFullGlobal_2023D3+HARVESTFullGlobal_2023D3/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2023D3_GenSimFull+DigiFull_2023D3+RecoFullGlobal_2023D3+HARVESTFullGlobal_2023D3.log

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
813e5ad
51dd54a
12077f8
851461b
5cb0fc2
43dfd1b
032ad55
3cfd184
4c378a3
f569019
941fa6e
a53847c
a03ca88
bc0c679
af763d9
ec5ff3c
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15889/15233/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15889/15233/git-merge-result

kpedro88 · 2016-09-17T15:59:34Z

Looks like an IB issue, let's try again with the new one

kpedro88 · 2016-09-17T15:59:47Z

@cmsbuild please test

cmsbuild · 2016-09-17T16:00:22Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/15241/console

cmsbuild · 2016-09-17T17:50:20Z

-1

Tested at: a19eefb

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15889/15241/summary.html

I found follow errors while testing this PR

Failed tests: RelVals

RelVals:

When I ran the RelVals I found an error in the following worklfows:
20824.0 step3

runTheMatrix-results/20824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2023D3_GenSimFull+DigiFull_2023D3+RecoFullGlobal_2023D3+HARVESTFullGlobal_2023D3/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2023D3_GenSimFull+DigiFull_2023D3+RecoFullGlobal_2023D3+HARVESTFullGlobal_2023D3.log

kpedro88 · 2016-09-18T03:56:11Z

@cmsbuild please test

cmsbuild · 2016-09-18T03:56:14Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/15246/console

slava77 · 2016-09-22T18:48:58Z

On 9/22/16 11:07 AM, Kevin Pedro wrote:

@slava77 https://github.com/slava77 this is the best way to remove
|HBHEChannelInfo| from the event products, according to @igv4321
https://github.com/igv4321

OK, if you do not need them produced, it is even better.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#15889 (comment), or
mute the thread
https://github.com/notifications/unsubscribe-auth/AEdcbjb3pOFPmw6CivyclgMSWg2mxhdzks5qssPogaJpZM4J_ha_.

cmsbuild · 2016-09-22T19:36:30Z

+1
Tested at: 833fe0e
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15889/15339/summary.html

cmsbuild · 2016-09-22T20:24:42Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15889/15339/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

10424.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017NewFPix_GenSimFull+DigiFull_2017NewFPix+RecoFull_2017NewFPix+HARVESTFull_2017NewFPix

cmsbuild · 2016-09-22T20:37:33Z

+1
Tested at: 21e6fec
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15889/15343/summary.html

cmsbuild · 2016-09-22T21:47:57Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15889/15343/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

10424.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017NewFPix_GenSimFull+DigiFull_2017NewFPix+RecoFull_2017NewFPix+HARVESTFull_2017NewFPix

slava77 · 2016-09-23T01:23:52Z

Looking at jenkins output (done with CMSSW_8_1_X_2016-09-22-0900)

10024 has all hbhe flags disappear

otherwise, downstream reco objects appear to be unchanged. This is probably not so good for the flags, but it is good enough for the purpose of keeping the default old 2017 hcal setup unchanged.

slava77 · 2016-09-23T01:44:47Z

+1

for #15889 21e6fec

new workflows, including full HCAL 2017 setup (still XML, but apparently OK for now). The PR description and title has to be updated to describe what was added. The current default 2017 as in pre11 is essentially preserved (modulo the flag settings).
jenkins tests pass and comparisons with baseline show changes only in 2017 workflows in hcal flags (there is no cross-comparison between scenarios, which would have shown some more noticeable changes)
signoff for reco relies mostly on results of comparisons as of Monday (commit range in the latest version of this PR is 86cd2d7...909daed) which included what now would be a cross-workflow comparison of "2017" and "2017HCALdev" workflows (see HCAL 2017 Phase1 Workflows #15889 (comment))
- hbhe channel infos are now not saved, which should reduce the increase in RECO on disk by ~2/3.

slava77 · 2016-09-23T01:47:47Z

given a possibility to compare the 4 types of 2017 workflows even with the short matrix,
it may be more productive to merge this and then aim to fix up the DB geometry and related GT updates separately rather than converge on a more perfect setup.

kpedro88 · 2016-09-23T03:05:10Z

@slava77 updated the title and description

ianna · 2016-09-23T05:49:30Z

+1

ianna · 2016-09-23T10:52:56Z

@mmusich - just for the record: HCal 2017dev simulation geometry as in XMLFILE_Geometry_81YV8_Extended2017dev_mc is fine. Here is one built from XML vs DB one - they are identical:

mmusich · 2016-09-23T10:59:49Z

@ianna, thanks does this mean that #15889 (comment) had a bug in it?

bsunanda · 2016-09-23T13:25:45Z

@ianna I find strange results from the geometry in data base. I am using the xml geometry using the lines:
process.load('Geometry.CMSCommonData.cmsExtendedGeometry2017devXML_cfi')
process.load('Geometry.HcalCommonData.hcalDDConstants_cff')
process.load("Geometry.HcalEventSetup.hcalTopologyIdeal_cfi")
and compare with results from DB using the prescription from you
process.load("Configuration.StandardSequences.FrontierConditions_GlobalTag_cff")
from Configuration.AlCa.autoCond import autoCond
process.GlobalTag.globaltag = autoCond['phase1_2017_hcaldev']
process.load('Configuration.StandardSequences.GeometryDB_cff')

I do not see any difference in the HcalParameters as used in the test for HcalParameters, HcalSimNumberingConstants or HcalRecNumberingConstants. But there are large differences in the geometry values. The number of ValidId's are the same but the centers and the corners are widely different which cannot be due to rounding. I wonder what could have caused this,

ianna · 2016-09-26T09:00:23Z

@bsunanda - I've opened a separate issue #15985

cmsbuild added this to the Next CMSSW_8_1_X milestone Sep 17, 2016

cmsbuild added comparison-pending daq-pending dqm-pending operations-pending orp-pending pdmv-pending pending-signatures reconstruction-pending tests-pending labels Sep 17, 2016

cmsbuild added tests-started and removed tests-pending labels Sep 17, 2016

cmsbuild added tests-rejected and removed tests-started labels Sep 17, 2016

cmsbuild added tests-pending and removed tests-rejected labels Sep 17, 2016

cmsbuild added tests-started and removed tests-pending labels Sep 17, 2016

cmsbuild added tests-rejected and removed tests-started labels Sep 17, 2016

cmsbuild added the tests-started label Sep 22, 2016

cmsbuild added comparison-available and removed comparison-pending labels Sep 22, 2016

cmsbuild added comparison-pending tests-approved and removed comparison-available tests-started labels Sep 22, 2016

cmsbuild added comparison-available and removed comparison-pending labels Sep 22, 2016

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Sep 23, 2016

kpedro88 changed the title ~~HCAL 2017 Phase1 Workflow~~ HCAL 2017 Phase1 Workflows Sep 23, 2016

cmsbuild added geometry-approved and removed geometry-pending labels Sep 23, 2016

davidlange6 merged commit 831786b into cms-sw:CMSSW_8_1_X Sep 23, 2016

kpedro88 mentioned this pull request Oct 5, 2016

Fixes for HCAL premixing #16057

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HCAL 2017 Phase1 Workflows #15889

HCAL 2017 Phase1 Workflows #15889

kpedro88 commented Sep 17, 2016 •

edited

Loading

kpedro88 commented Sep 17, 2016

cmsbuild commented Sep 17, 2016

cmsbuild commented Sep 17, 2016 •

edited

Loading

cmsbuild commented Sep 17, 2016

kpedro88 commented Sep 17, 2016

kpedro88 commented Sep 17, 2016

cmsbuild commented Sep 17, 2016 •

edited

Loading

cmsbuild commented Sep 17, 2016

kpedro88 commented Sep 18, 2016

cmsbuild commented Sep 18, 2016 •

edited

Loading

slava77 commented Sep 22, 2016

cmsbuild commented Sep 22, 2016

cmsbuild commented Sep 22, 2016

cmsbuild commented Sep 22, 2016

cmsbuild commented Sep 22, 2016

slava77 commented Sep 23, 2016

slava77 commented Sep 23, 2016

slava77 commented Sep 23, 2016

kpedro88 commented Sep 23, 2016

ianna commented Sep 23, 2016

ianna commented Sep 23, 2016

mmusich commented Sep 23, 2016

bsunanda commented Sep 23, 2016

ianna commented Sep 26, 2016

HCAL 2017 Phase1 Workflows #15889

HCAL 2017 Phase1 Workflows #15889

Conversation

kpedro88 commented Sep 17, 2016 • edited Loading

kpedro88 commented Sep 17, 2016

cmsbuild commented Sep 17, 2016

cmsbuild commented Sep 17, 2016 • edited Loading

cmsbuild commented Sep 17, 2016

kpedro88 commented Sep 17, 2016

kpedro88 commented Sep 17, 2016

cmsbuild commented Sep 17, 2016 • edited Loading

cmsbuild commented Sep 17, 2016

kpedro88 commented Sep 18, 2016

cmsbuild commented Sep 18, 2016 • edited Loading

slava77 commented Sep 22, 2016

cmsbuild commented Sep 22, 2016

cmsbuild commented Sep 22, 2016

cmsbuild commented Sep 22, 2016

cmsbuild commented Sep 22, 2016

slava77 commented Sep 23, 2016

slava77 commented Sep 23, 2016

slava77 commented Sep 23, 2016

kpedro88 commented Sep 23, 2016

ianna commented Sep 23, 2016

ianna commented Sep 23, 2016

mmusich commented Sep 23, 2016

bsunanda commented Sep 23, 2016

ianna commented Sep 26, 2016

kpedro88 commented Sep 17, 2016 •

edited

Loading

cmsbuild commented Sep 17, 2016 •

edited

Loading

cmsbuild commented Sep 17, 2016 •

edited

Loading

cmsbuild commented Sep 18, 2016 •

edited

Loading