SIGMETRICS Procedures

BYLAWS / CACM / COOPERATION / DUTIES

(Adopted: 11/17/2009)

Preamble:

From time to time, ACM SIGMETRICS receives requests from other conferences and/or workshops for "in cooperation" status with SIGMETRICS. Typically, these events have some sort of conceptual overlap with our SIG, reflecting shared interests in the methodologies and/or applications of computer systems performance evaluation. The event organizers often view official endorsement from SIGMETRICS as a "brand" to enhance the profile of their event. Currently, we receive about 10-12 of these requests per year, and grant approval for 4-5 of them. The process is somewhat ad hoc. In October 2009, ACM requested each SIG to develop a policy that documents the criteria used by the SIG for approval of "in cooperation" events.

Principles:

There are several guiding principles that govern our approval of "in cooperation" events. First, there must be an obvious area of overlapping interest with our SIG. Second, the event must be of sufficient profile and quality to merit endorsement by the SIG. Third, there must be mutual benefits to the partnership arrangement. For the conference organizers, the benefits might include credibility, use of ACM/SIG logos for branding, SIG distribution mechanisms for publicity, and ACM Digital Library as an avenue for dissemination. For the SIG, benefits might include greater visibility for ACM SIGMETRICS, registration fee discounts for SIG members, and the availability of additional content in the ACM digital library. Last but not least, the partnership arrangement clearly needs to be for technical and professional reasons, not for financial gain. Specifically, ACM forbids "in cooperation" arrangements with for-profit organizations.

Evaluation Criteria:

The following questions are used to assess "in cooperation" requests. While no specific question or criterion is necessarily a deal-breaker, the overall set of questions establishes our evaluation criteria, and illustrates the guiding principles defined above.

1. What is the level of overlap, if any, with ACM SIGMETRICS?

SIGMETRICS is willing to endorse events covering topic areas of interest to SIGMETRICS, particularly if the topics are of emerging or growing interest, or offer opportunities for new application domains, international outreach, or broader receptor communities. If the topic is not germane to SIGMETRICS, then endorsement is unlikely.

2. What is the perceived quality of the event?

ACM SIGMETRICS is willing to endorse high-quality events that bring value to members of the performance evaluation research community.

For events that are already endorsed by ACM or other SIGs, approval is straightforward, provided the conceptual overlap with SIGMETRICS is evident.

For events with a prior track record, assessment is based on the review process (e.g., full papers refereed by a respected TPC), acceptance rate (e.g., 30% or lower), and past proceedings from the event (e.g., 8-10 page papers in ACM two-column format in the ACM DL or ICPS). Events with a respectable track record of quality are good candidates for endorsement.

For newly-created events, assessment is based on less tangible aspects, such as the organizers, their track record, TPC, review process, sponsors, and the expected size, relevance, quality, and impact of the event. Secondary issues such as dates, location, venue, publicity, and logistics may also be considered. Well-organized events with clear potential to establish themselves as a respected publication venue deserve the chance to be supported.

3. Does the event conflict with other SIGMETRICS events?

SIGMETRICS discourages partners from running events that compete with our flagship SIGMETRICS conference, or other SIG events. Note that "competition" could mean competition for attendees (e.g., overlapping conference dates, or similar dates in very different geographies), papers (e.g., similar dates for paper submission and review cycle, or unfortunate date alignments), people (e.g., conference organizers, TPC members, keynote speakers) or partners (e.g., industry sponsors, other SIGs).

If the event is clearly one that can raise the profile of our SIG, without compromising the success of our own conferences and workshops, then it is a good candidate to receive endorsement.

4. Who are the primary sponsors of the event?

SIGMETRICS is willing to endorse events sponsored by other professional organizations, provided that they deliver strong technical value to the performance evaluation community. Partnership strictly for financial reasons is forbidden.

If the primary sponsor is ACM or another SIG, then obtaining endorsement from SIGMETRICS is relatively easy, since the financial model for the event is well-understood. Similarly, IEEE and IEEE CS events are relatively well understood.

If the sponsor is a non-profit organization, documentation regarding the organization and its conference operations may be required.

If the sponsor is a for-profit organization, the request for "in cooperation" status will be declined.

5. What are the benefits to the conference organizers?

Event organizers requesting "in cooperation" status should indicate why this status is being requested, and what value it brings to their event. If finances are the main driver, the request will be declined.

6. What are the benefits to the SIG members?

Event organizers should explicitly indicate the benefits being offered to the SIG. One general benefit is greater visibility for ACM SIGMETRICS in new or broader communities. A more specific benefit is a registration fee discount for ACM/SIG members. Where possible, the event organizers should make the proceedings from their event available in the ACM digital library, perhaps via the International Conference Proceedings Series (ICPS). Events with no clear benefits to SIGMETRICS members are unlikely to be endorsed by the SIG.

Process:

Requests for "in cooperation" events should be directed to the SIG Chair via the ACM approval process, including TMRF (Technical Meeting Request Form) and budget plan. Requests should be submitted in a timely fashion, preferably 6-12 months in advance of the event. In most cases, a decision will be made within 2 weeks by the SIG Chair, either individually or in consultation with the SIG Executive Committee.

Quick Links

Conferences

  • SIGMETRICS is the flagship conference of the SIGMETRICS community.
  • Performance is the flagship conference of IFIP WG 7.3 and occurs jointly with SIGMETRICS once every three years.
  • IMC is at the intersection between SIGCOMM and SIGMETRICS, focusing on Internet measurement.
  • SenSys focuses on the design, implementation, and application of sensor networks.
  • Greenmetrics is a high-profile workshop that focuses on both designing sustainable ICT systems and using ICT to improve sustainability. Note: Greenmetrics will not take place in 2015 due to FCRC. The workshop will return in 2016.
  • MAMA is a high profile workshop that focuses on mathematical performance modeling tools.
  • ValueTools bridges the study of performance methodologies and the development of tools for performance evaluation.
  • ICPE (formerly WOSP) focuses on the intersection of software design and performance evaluation.

Recent News

Publications

  • Performance Evaluation Review (PER) is the SIGMETRICS quarterly review. PER is a refereed publication that features guest edited special issues from leading researchers in the field.
  • Other SIGMETRICS publications
    The ACM Digital library provides a convenient archive of all SIGMETRICS publications, including PER and conference proceedings.