module Puppet::Pops::Parser::SlurpSupport

This module is an integral part of the Lexer. It defines the string slurping behavior - finding the string and non string parts in interpolated strings, translating escape sequences in strings to their single character equivalence.

PERFORMANCE NOTE: The various kinds of slurping could be made even more generic, but requires additional parameter passing and evaluation of conditional logic. TODO: More detailed performance analysis of excessive character escaping and interpolation.

Constants

DQ_ESCAPES
SLURP_ALL_PATTERN
SLURP_DQ_PATTERN
SLURP_SQ_PATTERN
SLURP_UQNE_PATTERN: unquoted, no escapes
SLURP_UQ_PATTERN
SQ_ESCAPES
UQ_ESCAPES

Public Instance Methods

slurp(scanner, pattern, escapes, ignore_invalid_escapes) click to toggle source

Slurps a string from the given scanner until the given pattern and then replaces any escaped characters given by escapes into their control-character equivalent or in case of line breaks, replaces the pattern r?n with an empty string. The returned string contains the terminating character. Returns nil if the scanner can not scan until the given pattern.

    # File lib/puppet/pops/parser/slurp_support.rb
 67 def slurp(scanner, pattern, escapes, ignore_invalid_escapes)
 68   str = scanner.scan_until(pattern) || return
 69 
 70   return str unless str.include?('\\')
 71 
 72   return str.gsub!(/\\(\\|')/m, '\1') || str if escapes.equal?(SQ_ESCAPES)
 73 
 74   # Process unicode escapes first as they require getting 4 hex digits
 75   # If later a \u is found it is warned not to be a unicode escape
 76   if escapes.include?('u')
 77     # gsub must be repeated to cater for adjacent escapes
 78     while(str.gsub!(/((?:[^\\]|^)(?:[\\]{2})*)\\u(?:([\da-fA-F]{4})|\{([\da-fA-F]{1,6})\})/m) { $1 + [($2 || $3).hex].pack("U") })
 79       # empty block. Everything happens in the gsub block
 80     end
 81   end
 82 
 83   begin
 84   str.gsub!(/\\([^\r\n]|(?:\r?\n))/m) {
 85     ch = $1
 86     if escapes.include? ch
 87       case ch
 88       when 'r'   ; "\r"
 89       when 'n'   ; "\n"
 90       when 't'   ; "\t"
 91       when 's'   ; ' '
 92       when 'u'
 93         lex_warning(Issues::ILLEGAL_UNICODE_ESCAPE)
 94         "\\u"
 95       when "\n"  ; ''
 96       when "\r\n"; ''
 97       else      ch
 98       end
 99     else
100       lex_warning(Issues::UNRECOGNIZED_ESCAPE, :ch => ch) unless ignore_invalid_escapes
101       "\\#{ch}"
102     end
103   }
104   rescue ArgumentError => e
105     # A invalid byte sequence may be the result of faulty input as well, but that could not possibly
106     # have reached this far... Unfortunately there is no more specific error and a match on message is
107     # required to differentiate from other internal problems.
108     if e.message =~ /invalid byte sequence/
109       lex_error(Issues::ILLEGAL_UNICODE_ESCAPE)
110     else
111       raise e
112     end
113   end
114   str
115 end

slurp_dqstring() click to toggle source

   # File lib/puppet/pops/parser/slurp_support.rb
33 def slurp_dqstring
34   scn = @scanner
35   last = scn.matched
36   str = slurp(scn, SLURP_DQ_PATTERN, DQ_ESCAPES, false)
37   unless str
38     lex_error(Issues::UNCLOSED_QUOTE, :after => format_quote(last), :followed_by => followed_by)
39   end
40 
41   # Terminator may be a single char '"', '$', or two characters '${' group match 1 (scn[1]) from the last slurp holds this
42   terminator = scn[1]
43   [str[0..(-1 - terminator.length)], terminator]
44 end

slurp_sqstring() click to toggle source

   # File lib/puppet/pops/parser/slurp_support.rb
25 def slurp_sqstring
26   # skip the leading '
27   @scanner.pos += 1
28   str = slurp(@scanner, SLURP_SQ_PATTERN, SQ_ESCAPES, :ignore_invalid_escapes)
29   lex_error(Issues::UNCLOSED_QUOTE, :after => "\"'\"", :followed_by => followed_by) unless str
30   str[0..-2] # strip closing "'" from result
31 end

slurp_uqstring() click to toggle source

Copy from old lexer - can do much better

   # File lib/puppet/pops/parser/slurp_support.rb
47 def slurp_uqstring
48   scn = @scanner
49   str = slurp(scn, @lexing_context[:uq_slurp_pattern], @lexing_context[:escapes], :ignore_invalid_escapes)
50 
51   # Terminator may be a single char '$', two characters '${', or empty string '' at the end of intput.
52   # Group match 1 holds this.
53   # The exceptional case is found by looking at the subgroup 1 of the most recent match made by the scanner (i.e. @scanner[1]).
54   # This is the last match made by the slurp method (having called scan_until on the scanner).
55   # If there is a terminating character is must be stripped and returned separately.
56   #
57   terminator = scn[1]
58   [str[0..(-1 - terminator.length)], terminator]
59 end