U.S. NATIONAL INTERPRETATION PROCESS: WHAT IS IT ALL ABOUT?

[James Arnold <arnoldj1@saic.com>]

Note: I have taken off all of my “hats” and the following material
represents only my own opinions and not that of anyone or any
organization with which I have been or continue to be associated.

James Arnold
--------------------------------------------------------------------

GENERAL COMMENTS:

* U.S. National Interpretation Process: What is it all about?

I am concerned about the objectives underlying the U.S. National
Interpretation process, and more so that they are not clearly presented
in any public document that I can find.

* Where do the interpretation issues come from?

It seems as though interpretations are being influenced by earlier U.S.
evaluation paradigms.  For example, it seems as though a number of
interpretations generated in the context of TTAP have been adopted by
the CCEVS, though they may not be inappropriate due to changes in the
CCEVS.  Observing that a number of CCEVS management approved
interpretations were finalized in March 2000 and many are the result of
superceding interpretations finalized in 1999, one wonders about the
source of interpretations since there wasn’t even an approved NIAP
evaluation laboratory until August 2000.

* What are interpretations for?

What exactly do these interpretations mean?  I see that they are
effective on a particular date, without further explanation The
Interpretation Overview discussion on the NIAP website indicates that
interpretations could be used as explicitly stated requirements.  One of
the CCEVS handbooks suggests that the U.S. will develop interpretations
to be submitted to the international CC interpretation group.  I am also
aware that CCEVS Validators have informed evaluation teams that they are
subject to interpretations effective when the evaluation starts.  But
back to the point, what does this mean?

* Interpretations are most important to developers and not evaluations
labs.

Treating the interpretations as explicit statements may be the best
answer; though one wonders why resources should be expended on this
activity when there are so many other areas of the CCEVS that could
benefit from additional guidance or review.  Feeding the international
CC interpretation group certainly has value, but the most important
issues are those that result from efforts directly related to
evaluations.  Telling (with no previous public guidance) evaluation
teams that interpretations are required to be used (?) is not good since
it would typically be too late at the beginning of evaluation,
presumably after all TOE related evidence has been produced, at great
expense, by the evaluation sponsor.  Note that I, unfortunately, believe
this is consistent with the apparent objective of the CCEVS to work only
with evaluation labs, and primarily in the context of evaluations, but
this does not serve those who are investing the most toward the success
of any given evaluation.

* How is the CCEVS encouraging participating in the interpretation
process?

Basically, if the interpretation process is to have value, evaluation
sponsors and laboratories must be encouraged to participate.  This
includes making interpretation information easy to obtain/find,
explaining where interpretations come from, what problems they are
intended to address, and how they are expected/required to be used and
by whom.  To my knowledge, none of these issues is currently addressed,
and yet interpretations are apparently being approved, good or bad, at a
moderately fast rate.

* Interpretations may cause more problems than they solve.

I believe that failing to clearly and publicly explain the
interpretation process could cause far more serious problems than the
interpretations are intended to solve.  For example, there likely will
be impacts in the efficiency and cost of evaluation that will be
detrimental the overall success of the CCEVS.  These costs will be
realized in the form of added interpretation analysis, resolving
international recognition issues, rework due to misunderstanding or no
understanding at all, criteria debates, etc.

* Why is the U.S. promoting divergence from the international standard?

I am also concerned about U.S. divergence from the internationally
accepted CC standard in the form of interpretations.  Evaluation
sponsors that market products internationally are more interested in
mutual recognition than in any details or nuances of any evaluation
scheme.

* How do the interpretations work in the context of mutual recognition?

If an evaluation sponsor adopts interpretations of any evaluation
scheme, in many cases they will be using language that would be
considered “explicitly stated” in n-1 of the countries involved in
mutual recognition.  As such, it is clearly better to stick with the
requirements accepted by the majority.  Use of interpretations would
serve to confuse other schemes that intend to exercise mutual
recognition, since there would appear to be explicitly stated
requirements without the expected rationale for such requirements.
Furthermore, they would be even more confused to find the original CC
words rationalized as though they were explicitly stated (a possible
result of refusing to accept an interpretation).  Worse might be
inconsistency brought to all users of PPs and STs resulting from
conformance to a moving target (the CC and interpretations effective on
a given day).  Note that I believe that the U.S. cannot interpret any
assurance requirements, since this would effectively represent a
requirement explicitly stated in the U.S. and as a result it would not
be mutually acceptable under the rules of the CCRA.

   - How should a U.S interpretation be treated by a developer?  Should
it be treated as an explicit statement?

   - What happens if a U.S. developer chooses to use the original
requirement rather than the interpretation?  Should it be treated as an
explicit statement?  Will he CCEVS attempt to refuse/prevent the use of
the original statement, despite acceptance in the majority of the
participating nations?

   - What happens if a U.S. developer is using a PP that was approved
prior to an interpretation?  What if an affected requirement has an
operation to complete or is refined?  What if it needs a new
instantiation?  Would the rationale need to claim the PP requirement as
explicitly stated, even though it wasn’t when it was evaluated?

* The interpretation process should allow the definition of
alternatives.

A potential benefit of an interpretation would be to provide an
alternative, rather than a change.  Hence, the rational for what would
otherwise be an explicitly stated requirement would be satisfied by the
existence of the interpretation.  However, the original requirement
should continue to be valid since it is still accepted in the majority
of participating nations.

* Interpretations of meta-criteria should broaden and not restrict
applicability.

Given that the CC represents meta-criteria (that is, criteria to build
criteria), I think that the objective should be to broaden the potential
application of the criteria.  This could serve to improve evaluation
efficiency by reducing rationale burden for PP and ST authors by
referencing interpretations.  However, there are examples of approved
interpretations that both broaden and further restrict the potential
application of the criteria.  I think the latter is inappropriate for
meta-criteria, while it may be appropriate in the interpretation of the
content of PPs and the CEM.

To conclude my general comments on the interpretations process, I
recommend that the CCEVS should work to produce effective, public
guidance that addresses these and many other important questions.  I
furthermore offer that this should be done cautiously with a goal of
improvement, not in the strength of requirements, but in the efficiency
and overall cost of obtaining evaluations.

Note that as I constructed the comments that follow, I came to the
additional conclusion that perhaps interpretations are being overused.
They are being used to define new components as well as to express CCEVS
policy and guidance.  It seems that those things could be more
effectively dealt with outside the interpretation process.

SPECIFIC COMMENTS:

My specific comments first address the interpretation outline and then
generally address the set of already approved interpretations (note that
I am unable to say whether I or any other outside parties have had an
opportunity to comment on these interpretations in the context of the
CCEVS).

* What exactly is being interpreted?

>From what I can tell, many of the interpretations simply change lots of
words in the various CC volumes, but it is not always clear there is
actually an interpretation.  Regardless, some of the larger
interpretations change a lot of words in multiple CC volumes, and in
some cases words that were already changed by another interpretation.
The net result is a complex problem of reconstructing the modified
versions of the CC.  I wonder if there is intent to provide interpreted
CC volumes to help alleviate some of the burden?

Additionally, a number of the approved interpretations are not actually
interpreting existing criteria, but rather are simply introducing new
components.  This is quite simply the development of new criteria and
not interpretation at all.  (See I-0348, I-0352, I-0363)

* How do I find an interpretation?

The problem here is that the interpretation website allows
interpretations to be accessed by component.  However, there are
interpretations, for example that change definitions in Part 1, that
have an impact on a number of components, and these changes are not
associated with the components that are being changed.  I think that
most users would be surprised to find that a change to Part 1 would
effectively change a component in Part 2. I recommend that the
association include indirect as well as direct changes/effects.

* What is the impact?

A number of accepted interpretations claim that there is a negligible
impact on on-going evaluations.  This seems a silly statement for March
2000, when no CCEVS evaluation laboratories existed.  All of the
approved interpretations claim “negligible” impact.  It is not clear
what this is based on.  For example, I-0424 causes TOEs that satisfy
FPT_SEP.3  to not implicitly satisfy FPT_SEP.2.  While this may have not
meant much when written, it certainly could cause a problem with an ST
claiming conformance to a PP.  In any case, this phrase comes from an
earlier paradigm where it was assumed the interpretation was necessary
and the impact was implicitly associated with expected evaluation
problems.  I think in the new paradigm the value of the interpretations
is not clear and I recommend that “positive” impacts be included.  After
all, if the interpretation truly has a “negligible” impact, one wonders
why it was created.

An additional issue here is the complexity of interpretations (as
mentioned above) and the proliferation of interpretations, both of which
have negative impacts.  Complex interpretations may be harder to use
than the original text and combinations may introduce unforeseen
problems.  Proliferation of interpretations increases the burden of the
users of the criteria, makes the criteria target move, and serves to
confuse users about what they need or what they are getting.

COMMENTS ON ACCEPTED INTERPRETATONS:

I-0339: Assurance Of RVM Is By Testing And Design Analysis

I’m a little confused by the nature of this interpretation.  It seems to
justify itself by claiming that most SFRs can be verified exclusively
through testing.  However, each of the EALs defines a whole set of
assurance requirements; all of which include both testing and
development evidence evaluation, to demonstrate that all SFRs are
addressed by the TOE.  Note that there are likely to be aspects of most
SFRs that cannot be demonstrated by testing alone, and RVM is not
especially outstanding in this regard.  Basically, I think this
interpretation is unnecessary since the developer must provide a design
argument (based on ADV requirements) that the SFRs are satisfied.  Note
that I am also confused about the indication that the CEM should also be
interpreted – there are other interpretations that interpret more than
one volume of the CC already – why hasn’t that interpretation been
included here?

I-0348: Audit Data Loss Prevention Method May Be Site-Selectable

It is not clear this is an interpretation.  Rather it appears simply to
be the definition of a new component.  It is also not clear it is useful
since it simply adds a new element to address the default handling on
overflow, which could have been specified in the assignment in FAU_STG.4
in any case.  Note that if I am correct, this interpretation is in
conflict with the notion presented in I-0424 regarding hierarchical
components.

I-0362: Scope of Permitted Refinements

Part 2, para 81, already states “Refinement shall only further restrict
the set of possible acceptable functions or mechanisms to implement the
requirements, but never increase it.”  I think this pretty much covers
the interpretation.

I-0364: Application Notes In Protection Profiles Are Informative Only

While the CC could be clarified as suggested, I don’t think that
anything already existing in the CC explicitly indicates that
application notes may be normative, while it does so for other
sections.  I think that this is indicative of guidance that should be
provided by the CCEVS and perhaps documented in the CEM, but doesn’t
necessitate an interpretation.

I-0385: Identification Of Standards

I think the CCEVS would be a lot better off to simply not support any
claims to external standards.  I am aware of challenges regarding
compliance to Protection Profiles based on the CC, now we are talking
about accepting claims about standards that may not even be based on the
CC.  Does the evaluation team have to ensure that the external standard
makes sense as seems to be required of existing CC-based standards?  I
recommend that a policy statement indicating that the CCEVS doesn’t
verify claims outside the scope of the CC replace this interpretation.

I-0405: American English Is An Acceptable Refinement

According to Part 2 (para 81) a refinement can only restrict the
possible set of acceptable functions or mechanisms.  Hence, changing
language does not technically qualify as a refinement.  Perhaps that is
why the CC introduces the notion of “special” refinement with a zero net
change.  Regardless, I think that this interpretation is weak since it
supports mixing languages (i.e., UK and US English) in a single PP or
ST.  I think the interpretation should be replaced by a CCEVS policy
statement allowing PPs and STs to be translated into any acceptable
language (as I’m sure other schemes do) with the stipulation that the
result is a single language for consistency.

I-0423: Some Modifications To The Audit Trail Are Authorized

Deletion of audit records withstanding; it is a bad idea to allow any
modifications to audit records (there could be serious legal
implications).  Perhaps this is a case where a new hierarchically
inferior component should be added.  I wonder how the interpretation
group is determining which of the informative annex or the normative
requirement statement is correct.

I-0424: FPT_SEP.2 And FPT_SEP.3 Are Not Hierarchical

I think a mistake was made here.  I believe that the phrase “A component
is hierarchical to another if it offers more security”, which serves as
the basis for the result, is simply incorrect.  Limited support for my
statement can be found in Part 2 (para 63), but I believe that it is
commonly understood that satisfying a hierarchically greater component
implies satisfaction of a hierarchically inferior component.  This would
be true for the pair of requirement components addressed in this
interpretation.  The definition of hierarchical should be changed to
“equal or more security”.  In this case, it is possible to make
FPT_SEP.2 equal to FPT_SEP.3, but it is not necessary.  It is quite
possible that FPT_SEP.2 can require less than, but never more than,
FPT_SEP.3.  I also have to wonder about this example because there are a
great number of other similar examples (e.g., FIA_UID.1/UID.2,
FIA_UAU.1/UAU.2).  Changing the hierarchical mapping in FPT_SEP serves
to solve no problem that I can think of.  Unless, or course, there is
concern that a developer may claim FPT_SEP.2 when they could have
claimed FPT_SEP.3 instead.  I recommend replacing this interpretation
with a clarification of the definition of “hierarchical”.

I-0426: Content Of PP Claims Rationale

I am wondering how the interpretation group is able to determine whether
the words of Part 1 or those of Part 3 are correct.  Also, since
explicitly stated assurance requirements are not recognized under the
CCRA, it seems that this simply introduces an added burden on only
evaluations in the U.S. Scheme.