Minutes of the INTSERV WG Meeting 40th IETF, Washington DC, USA Monday 9:30-11:30 Meeting notes by Rob Cheng and Allison Mankin, thanks! Edited by John W. John Wroclawski began the meeting with a brief presentation of the agenda and proposed goals. Goals of the meeting: 1) Understand what problem or problems each proposal is trying to solve. - Internal Class of Traffic preference within an ISP - Customer QoS Control (end-to-end, end-to-middle, access link) - Int-Serv/RSVP Aggregation & Scaling - Hierarchical Coding Support (audio/video) 2) Understand areas of commonality & disagreement in the proposals 3) Work towards agreement on a plan for moving forward Agenda: - Presentations from: Dave Clark (for himself, Kathie Nichols, Lixia Zhang, and Van Jacobson) Paul Ferguson Juha Heinanen Jean-Yves Le Boudec Zheng Wang Kilkki Kalevi Steven Blake Speakers were asked to describe what they thought should be done now, what might be done later, and what was not necessary, as well as describing their own proposals. - Discussion - Presentation by Dave Clark "A combined approach to Differential Service in the Internet" Slides from Dave's presentation are available at http://diffserv.lcs.mit.edu/IETF40/ddc_is_dec97.pdf http://diffserv.lcs.mit.edu/IETF40/ddc_is_dec97.ps http://diffserv.lcs.mit.edu/IETF40/ddc_is_dec97.ppt Notes from Dave's talk: Goal of talk is to describe a framework that can help distinguish between _mechanisms_ in the routers and the range of _application_services_ that can be built from them. Specifically, describe a combination of the two schemes ("Van's" & "Dave's"). Bottom line: they can be used together. Proposal for action: Identify & agree on some _mechanism_ (routers & hosts) - criteria: general, implementable - specific proposals - standards track RFC - informational RFCs (illustrations) - how to use bits (standards) Definitions: What is inside the box (router/host) is called a _mechanism_; the mechanism has a specific (hop-by-hop) _behavior_. The user of the network (could be an end-user or a subsidiary network) obtains an (end-to-end) _service_, which can be obtained by using the mechanisms following certain _rules_. Mechanisms can operate, and rules can apply, at several places within the network: 1) router: mechanism controls scheduling during congestion. 2) boundaries between networks: mechanism verifies usage patterns, tags, shapes, logs, enforces rules. 3) end nodes: generate/sink traffic, mark traffic according to profile Two points: 1) The (hop-by-hop) _behavior_ of the mechanism is NOT the same as the (end-to-end)_service_ given to the user. 2) User-visible services are NOT the ONLY objective (e.g. network mgmt.) Idea/concept: Profile meters at the edge, inside a network cloud no checking. Specifics - some examples of using mechanisms according to rules to obtain a service: Router Profile Meter Resulting Service Q w/ higher Token buffer w/ Emulate fixed capacity priority constant rate circuit (not same as the (Van) output mechanism) Q w/ RIO Tags packets Allocated capacity profiles dropper In/Out based on (Dave) Profile Same Host tagging of Application-controlled hierarchical video drop preference Same Classification based on Favoring of selected traffic addresses during overloag (not e-2-e, but local intra-ISP management) (note the last three service examples result from different use of the same mechanism) Conclusion for mechanisms: - the two mechanisms can both be used to do useful things. - there is demand for services that can be built from either - so put in both Point: Need to more tightly specify a per-hop mechanism, in order to build a _range_ of services from it. Might be able to be less specific if the mechanism is only to be used to provide one service, but that is not (Dave's) goal. (ed: a later discussion spent some time on the distinction between mechanism and behavior; and whether it was possible to sufficiently tightly specify a behavior in a mechanism-independent way.) Multiple roles of traffic meters: - meter & router mechanisms work in concert to provide service - meters enforce rules (diff. rules at diff. parts of the network) - meters between providers may not need to care about appl. specific issues (edge will take care of it) Two views of how to use bits in packet headers: 1) bits describe some end-to-end goal (e.g. "lower delay") 2) bits _select_ or _control_ per-hop mechanism(s) used as building blocks. (Dave's proposal) ADVANTAGES of 2) - many more possible end-to-end services than per-hop mechanisms -> it's more flexible if don't assign bits to services - new services implemented all the time; don't want to standardize each - don't want/need to know the service in the "middle" of the net if the mechanism is working properly DISADVANTAGES of 2) - if network is very heterogeneous, may want to build the e2e service using different mechanisms at different points (in which case it would be useful for arbitrary boxes in the middle of the network to know what -service- the packet was receiving, so it could choose the right mechanism. Otherwise, service info must be carried some other way in this case.) - some end-to-end goals (services) may be so universal that it would be efficient/helpful to reserve a bit-pattern to identify them. "Aggregate & Forward" instead of "Segregate & Bill" model. Framework: - mechanism selection (priority queue, RIO queue, others) per-packet control over what mechanism does (priority routing for priority queue, appl. control (in-out bit) for RIO queue) - use TOS value to select mechanism; TOS bits _may_ change in the middle of the network - 1 level of descrimination (packet control of the mechanism) probably sufficient (In/Out); more bits may be useful in certain contexts, such as a military net where load could be shed in stages. - Presentation by Paul Ferguson "Simple Differential Services: IP ToS and Precedence, Delay Indication, and Drop Preference" Slides from Paul's presentation are available at http://diffserv.lcs.mit.edu/IETF40/pf_is_dec97.pdf http://diffserv.lcs.mit.edu/IETF40/pf_is_dec97.ps http://diffserv.lcs.mit.edu/IETF40/pf_is_dec97.ppt Notes from Paul's Talk: - High-level message is that we need something simple and easy to solve the near-term problem. - we should focus on the _semantics_ of the TOS, IP Precendence fields. (implementation details at upper level - RIO, weighted RED, Flow Red,... should be saved for FUTURE discussion...maybe leave it to vendors?) - existing semantics: RFCs 701(IP Prec), 1349(TOS) (not used) - new semantics suggested in (Paul's) draft (draft-ferguson-delay-drop-00.txt) - IP Precedence: encodes packet's drop preference - TOS: encodes packet's delay sensitivity - Minimized encoding w/ 2 bits possible - how drop preference might be used: - Pkt is marked (by either end system or 1st hop router) - Class granularity is fairly robust - actual implementation of preferential drop may be a done using a number of diff. mechanisms (work out implementation details later) Again, define the SEMANTICS, not the implementation details! - how delay might be used: - Pkt is marked - Delay class -> CBQ queue - Delay indication provides NO a priori knowledge of e-2-e delay, jitter, etc. - soft suggestions on how to handle delay sensitive and non delay sensitive traffic at intermediate nodes. - what problems should we be trying to solve? - SIMPLE traffic differentiation - SEMANTICS which interoperate even if different mechanisms are used to implement them in different routers, etc. - SCALEABLE mechanism - work on NOW: - bit semantics (allow implementors to deal w/ mechanisms and mapping to service discipline) - work on LATER: - explicit e-2-e signalling? - RSVP/Diffserv. internetworking? - Controlling non-adaptive flows - OUT OF SCOPE: - "creeping featurism" - "convoluted complexity" - modification of existing transport protocols - explicit e-2-e signalling? - Questions Paul would like to raise for discussion: - Does ECN really provide any intrinsic value? - is BW brokering worthwhile? overly complex? realistic in short-term? - will flow-based semantics scale? - is "lightweight" RSVP really needed? - Presentation by Juha Heinanen "Use of the IPv4 ToS Octet to Support Differential Services" Slides from Juha's presentation are available at http://diffserv.lcs.mit.edu/IETF40/jh_is_dec97.pdf http://diffserv.lcs.mit.edu/IETF40/jh_is_dec97.ps http://diffserv.lcs.mit.edu/IETF40/jh_is_dec97.ppt Notes from Juha's Talk: Problem: ISP Requirements - provide customer some _minimum_guarantees_ - delay insensitive - delay sensitive - customer should be able to indicate "importance" of packets, and, if available (net lightly loaded), get more than the minimum BW, BOTH for delay insensitive AND delay sensitive traffic. What should be standardized? - what _Types of Service_ are to be supported? - how many levels of drop preference are needed to indicate packet importance within a Type of Service Proposal: - 2 TOS's: - assured normal delay (Dave's) - assured low delay (Van's) - start with 2 levels of drop preference (In/Out) Possible Extensions: - more Types of Service (premium? network control?) - relationship between low delay assured service and premium service? - mechanism vs. services? - more levels of drop pref? - other uses of the TOS octet are OUT OF SCOPE. (e.g., ECN is a transport issue) - Presentation by Jean-Yves Le Boudec "Scalable Reservation Protocol" Slides from Jean-Yves' presentation are available at http://diffserv.lcs.mit.edu/IETF40/jyb_is_dec97.pdf http://diffserv.lcs.mit.edu/IETF40/jyb_is_dec97.ps Notes from Jean-Yves' talk: OBJECTIVE: _Scaleable_ Reservations (no per-flow state info. in routers) SRP offers "sticky network" service, best effort -> reservations may fail Four key technical features: 1) routers keep track of _total resources reserved_ only. 2) routers learn about reservations "on the fly" 3) *end systems cooperate* 4) no out-of-band signalling Protocol uses packets of 3 types (2 bits needed in header): - REQUEST - RESERVED - BEST-EFFORT How does it work? - sources of reserved-resource traffic can send REQ pkts to request reservation or RES packets if reservation is in place. Source learns that reservation is in place from destination; see below. - routers may discard/downgrade REQ but not RES pkts - receiver observes sucessful transmission rate, reports to source - source can then send RESERVED packets at rate given by receiver. (may use traffic shaping to do this) - routers, etc use measurement based admission control. - Estimation method for resource usage given in slides - SRP Requires trusted end-system behavior (but is first-hop router policing and verification possible?) - Presentation by Zheng Wang "Scaleable Service Allocation for the Internet" Slides from Zheng's presentation are available at http://diffserv.lcs.mit.edu/IETF40/zw_is_dec97.pdf http://diffserv.lcs.mit.edu/IETF40/zw_is_dec97.ps Problems with current Internet: - Best Effort - No isolation - Encourages greedy behavior - Difficult to quantify services Problems w/ per-session reservations - accounting and inter-ISP settlements not ready yet - per-session reservation not useful/practical for all apps - traffic characterization is HARD - administrative overhead is important - fine granularity (RSVP 5-tuple) means complexity diff-serv "in between" best effort and per-session reservations (BE: no isolation, DS: aggregated resource management, Per-session reservation: per flow isolation) What are we trying to achieve? - traffic isolation at a scaleable granularity - long-term service allocation for aggregated traffic - start with intra-domain protocols/mechanisms, expand to interdomain Important Questions: - diff-services over single path (specific endpoints), or anywhere in ISP? - strict or adaptive profile? - edge or bottleneck profile? (control resource usage at edge or bottleneck?) Problems w/ single profile-based tagging: edge control - don't know where the flow is going - what if many go to the same place? - what about reverse-path traffic (destination based control) Zheng's proposal: User-Share Based Differential service You can buy - a slice of a link - a virtual ISP You get (contract) - Guaranteed minimum bw over a specific path or anywhere within ISP. - Proportional sharing of excess bw. System considers two things: - ID of user (who) - Size of slice Application/user can select policy, packet preference within share Deployment - proportional share scheduling: Deficit round-robin or similar mechanism: O(1) Complexity, well understood - distribution of user share information (static, long-time-scale) use SNMP or other network management capability, at least to start. - fully incremental deployment is practical - Presentation by Kilkki Kalevi "Why Should we Reserve Three Bits for Drop Preference" Slides from Killki's presentation are available at http://diffserv.lcs.mit.edu/IETF40/kk_is_dec97.pdf http://diffserv.lcs.mit.edu/IETF40/kk_is_dec97.ps http://diffserv.lcs.mit.edu/IETF40/kk_is_dec97.ppt Notes from Killki's talk: Goals of Diffserv work: - avoid resource reservation - scalable (but he said speed, not size) - support range of services best effort, guaranteed, realtime, not, connectionless and connectionoriented flows - smooth evolution Why 3-bits for drop preference? 3 DP bits - Is NOT an appendage to system w/ RSVP, guaranteed, controlled load, and Best Effort - Does not directly map into packet loss priorities. (Does NOT mean that ISP tries to offer 8 different services with different loss probabilities). - is instead a TOOL for network operator to implement featuers (lower level mechanisms) - no need to keep trak of flows in network node, becuase information is in every packet - with delay bits, makes it possible to integrate real-time and non-real-time services. - more dynamics for traffic variation than one DP bit. 3 bits: granularity, flexibility, dynamics, scalability avoids need for signalling/reservations - Presentation by Steven Blake "Packet Marking" Slides from Steven's presentation are available at http://diffserv.lcs.mit.edu/IETF40/sb_is_dec97.pdf http://diffserv.lcs.mit.edu/IETF40/sb_is_dec97.ps http://diffserv.lcs.mit.edu/IETF40/sb_is_dec97.ppt Notes from Steven's talk: Model is services implemented by components: - traffic profile (profile meter) - router mechanisms - admission control (for diffserv, must consider -scope- of assurance) - host congestion control Intserv and Diffserv differ in service allocation policy: Intserv/RSVP dynamic immediate hop-by-hop Diffserv static long-term domain-wide (sees this as an artifact of lack of customer signalling, not necessariliy need for aggregation of state.) Service assurance level of assurance possible is dependent on stat-mux gain that allocator is trying to achieve - dependent on timescale - dependent on space scale (explicit path vs domain-wide) Static, domain-wide services must be conservatively allocated (for high assurance) because of lack of knowledge Dynamic or destination specific services can be allocated more agressively Level/degree of connectivity/service for NON assured/premium traffic important (deliver user's expectation for BE service) Evolution hypothesis 1 - wide range of allocation time/space scales will be needed, offered) hypothesis 2 - packing efficiency can be imprioved if resources are explicitly provisioned for each time-pace allocation (i think this means that if you use whatever detailed information is available you can pack better) conclusion - router differentation model should evolve to a variant of class-based queueing. - hierarchy of bandwidth and sharing models - providing minimal assured resources for each tier. This was the last presentation. The meeting moved to discussion. As a discussion starter, John Wroclawski presented a strawman proposal for a work plan and timeline for moving forward: ++ Now) Start work from the model of bits in the packet header selecting a queueing mechanism (or "action" or "behavior") taken at a router and a parameter or argument to the action. Begin writing a framework document: - description of problems different people are trying to address - examples showing how approach can solve these problems - description of components (router functions, traffic meters, etc.) to be developed now - description of components of interest but not considered for first round of work. ++ ~Late Feb. 98) Framework document nearing completion. Hold an interim meeting to discuss the framework document, and to begin work in two technical areas: 1) Begin considering details of bit allocation for diffserv within the TOS byte. Need to consider: - number of router actions to support and number of bits for an action parameter or argument (is one enough?) - Which header bits can be changed in the middle of the network. - deployment problems caused by current uses of these bits. 2) Begin selection and description of a small number of initial standard router actions - Priority queueing, RIO, Weighted queues, etc. - need to reach some agreement on how and how precisely these actions should be described. ++ April 1998 IETF Present and discuss work in progress in the two areas above. ++ August 1998 IETF Present and discuss documents resulting from work in the two areas above. Goal is that initial versions of these documents be nearing publication. ++ ~August/September 1998 Review status of longer-range diffserv research and implementation outside the IETF, and consider whether IETF WG should do something more based on that research. Examples of topics that might be appropriate to take on at this time include - Interdomain diffserv and needed protocols * Bandwidth Brokers * RSVP Aggregation and diffserv/intserv relationship - Receiver control of differential service * Use of ECN bits in diffserv ++ December, 1998 IETF Present initial work on any new topics taken on by the WG as a result of the August work review. A general discussion then began. (ed: I tried to mark the start of a thread with a Q and the followon comments with A's even though the discussion wasn't strictly question-and-answer...) Q (Mark Garrett): What's the service model: are concrete mechanism mapped to the concrete applications they are to support? Q (Jim Boyle): Are you talking about standardizing queueing algorithms? A (Dave Clark): There's a spectrum. Intserv specified hop-by-hop router behaviors that provided a single end-to-end service, kept spec as loose as possible. Here I'm suggesting specifying a set of building blocks that can be used for new purposes over time - may need tighter spec. an algorithm in C code-------------------------very loose specification | (like controlled load) | somewhere in the middle here (like TCP congestion control algorithm development - ) A (Paul Ferguson): I disagree with this standardization; keep it SIMPLE! - don't need to solve all the problems of the word - don't tell vendors how to do queueing/routing implementation - define a few mechanisms LOOSELY - do we really want to standardize the mechanisms? A (Dave Clark): It's easy to say simple-sounding things, like "a drop-level corresponds to a particular queue threshhold to start dropping", and find underconstrained specification results in wildly unworkable behaviors. "Simple" arguement is important but should not go to the point of making Building Blocks useless. The hard question for me is - how much must we constrain semantics? -> get the right BEHAVIOR Q (Christian Huitema): We should not assume that once we write a spec that people will do the right thing, and therefore it is going to go right out there. Need market forces to make it work! What is in interfaces that market forces can play on: 1. bits 2. contract types NOT the queues,interior of routers.... WG needs to find technical mechanisms that allow market forces to play. A (Jon Crowcroft) difference between intserv and diffserv is that diffserv is after we got smarter about pricing. Result of Brian's BOF. Other part of the diffserv origins is the aggregation problem, and we use the bits there again, though it's very different. Details might be wanted at edges [+ cloud borders] only. A (Ed Elleson): diffserv already in commercial nets (SNA) - priority and bw allocation to prevent starvation. Consider deployment of diffserv a migration problem, and include customer bw alloc, which Dave has done but some others not. Use mechanisms already in routers for both edges and cores, because networks won't have profile meters around them everywhere quickly. Review what we can do now and make a migration part of our thinking. Q (Matt Mathis) I question the use of the word "mechanism" Do you really mean "a specific implementation/algorithm" or "a detailed behavior spec"? I think we'd really like a hop-by-hop _behavior_ spec for routers and traffic meters, but we don't quite know how to do it: thus we will spec mechanisms. - hop-by-hop behavior spec hard -> cheat by using implementation - fine line between mech. and services, but don't have the luxury of ignoring services <- they must be USEFUL so that you can charge extra for them. A [ Many nods of agreement] A (Jim Boyle) Hard to look at services directly, only examples. Q (Yoram Bernet, Microsoft): background points for emphasis - don't lose sight the value of RSVP as a signaling protocol from applications. Large numbers of hosts will be able to do this [RSVP is big in WinNT 5.0]. Hosts might also set diffserv bits based on feedback from RSVP. Q (Mike O'Dell): as a person building net services, not thinking about how to think about doing it, I have several points: nothing about what's inside the net makes sense. I want two things: - Grade of service, with small number of gradesT - Traffic type, early in packet, for a small number of application notions, such as Packet Voice. Don't tell me what to do, instead tell me what the user in some broad simple sense is looking for! Mental image is like a mime-type and a graded service within the type, 2 bits worth would be useful. Traffic types: probably need 16 bits. Use flow-id in IPv6 or reencapsulate in IPv4. Apps can easily say these categories. A (Allyn Romanow): grade of service _and_ type, why? A (Mike O'Dell) because they're orthogonal. E.g., highly important 6.9K voice versus don't care 6.9K voice. A (Dave Clark): I'm uncomfortable w/ the bits in header that specify application classes: - open-ended, unbounded classes? always new ones, how do you keep up? - what happens if I lie to get better performance? (call e-mail video). Is there a contract or pricing scheme that makes me set them to what I'm really doing? - Do you really want content identification in packets? Many in business land do want the packet type, so that tariffs can be charged by price. Usage fees for various materials being sent across the net, ISPs having to give accounting. A (Mike O'Dell): turning knobs to get your behavior...some of them will make your $ signs bigger, so that's OK. Sensitive to DDC's second concern, but it's a very limited subcase of the general problem. It doesn't go away if you don't type packets. Q (Wil Leland): multiple models in play here. Need some commonality enhancement. Q (Jack Houldsworth): ISO ECTS joint-authoring work on similar problems with ITU-T. Will send pointer to list, but it's an ID. Feel free to ignore its mcast contents. Q (Yoram Bernet): (to DDC) Don't assume end-systems are trusted. End-systems knowing/saying packet (application?) type is not what I had in mind either. I want something more abstract than a type tag. Q (Sally Floyd): Could talk about mechanism or whatever for some time. Propose to ratify John's schedule and milestones. We may have more common agreement than the last 1/2 hour might suggest. At this point the meeting was past closing time. The IESG Area Director asked for a closing comment: (Scott Bradner): We need help in deciding how this goes next. We have a number of mechanistic ways to proceed, from declaring it out of IETF bounds, to charting a new WG, to rechartering this one, to baying at the moon...Please catch Scott as the AD and give your opinions.