Commit Graph

58 Commits

Author SHA1 Message Date
Wojciech Mitros
88e750f379 idl-compiler: allow fields of type utils::chunked_vector
The utils::chunked_vector has practically the same methods
as a std::vector, so the same code can be generated for it.

Signed-off-by: Wojciech Mitros <wojciech.mitros@scylladb.com>
2021-01-13 04:09:18 +01:00
Pavel Solodovnikov
3a91f1127d idl: move enum and class serializer code writers to the corresponding AST classes
Expand the role of AST classes to also supply methods for actually
generating the code. More changes will follow eventually until
all generation code is handled by these classes.

Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2020-12-22 23:23:12 +03:00
Pavel Solodovnikov
edf9ccee48 idl: extract writer functions for write, read and skip impls for classes and enums
Split `write`, `read` and `skip` serializer function writers to
separate functions in `handle_class` and `handle_enum` functions,
which slightly improves readability.

Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2020-12-16 20:33:55 +03:00
Pavel Solodovnikov
8049cb0f91 idl: minor fixes and code simplification
* Introduce `ns_qualified_name` and `template_params_str` functions
  to simplify code a little bit in `handle_enum` and `handle_class`
  functions.
* Previously each serializer had a separate namespace open-close
  statements, unify them into a single namespace scope.
* Fix a few more `hout` -> `cout` argument names.
* Rename `template` pattern to `template_decl` to improve clarity.

Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2020-12-16 19:32:08 +03:00
Pavel Solodovnikov
0de96426db idl: change argument name from hout to cout in all dependencies of add_visitors fn
Prior to the patch all functions that are called from `add_visitors`
and this function itself declared the argument denoting the output
file as `hout`. Though, this was quite misleading since `hout`
is meant to be header file with declarations, while `cout` is an
implementation file.

These functions write to implmentation file hence `hout` should
be changed to `cout` to avoid confusion.

Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2020-12-16 19:32:03 +03:00
Pavel Solodovnikov
0defb52855 idl: fix parsing of basic types and discard unneeded terminals
Prior to the patch `btype` production was using `with_colon`
rule, which accidentally supported parsing both numbers and
identifiers (along with other invalid inputs, such as "123asd").

It was changed to use `ns_qualified_ident` and those places
which can accept numeric constants, are explicitly listing
it as an alternative, e.g. template parameter list.

Unfortunately, I had to make TemplateType to explicitly construct
`BasicType` instances from numeric constants in template arguments
list. This is exactly the way it was handled before, though.

But nonetheless, this should be addressed sometime later.

Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2020-12-16 19:31:57 +03:00
Pavel Solodovnikov
0cc87ead3d idl: remove unused functions
Remove the following functions since they are not used:
* `open_namespaces`
* `close_namespaces`
* `flat_template`

Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2020-12-16 19:31:51 +03:00
Pavel Solodovnikov
bea965a0a7 idl: improve error tracing in the grammar and tighten-up some grammar rules
This patch replaces use of some handwritten rules to use their
alternatives already defined in `pyparsing.pyparsing_common`
class, i.e.: `number`, `identifier` productions.

Changed ignore patterns for comments to use pre-defined
`pp.cppStyleComment` instead of hand-written combination of
'//'-style and C-style comment rules.

Operator '-' is now used whenever possible to improve debugging
experience: it disables default backtracking for productions
so that compiler fails earlier and can now point more precisely
to a place in the input string where it failed instead of
backtracking to the top-level rule and reporting error there.

Template names and class names now use `ns_qualified_ident`
rule instead of `with_colon` which prevents grammar from
matching invalid identifiers, such as `std:vector`.

Many places are using the updated `identifier` production, which
is working correctly unlike its predecessor: now inputs
such as `1ident` are considered invalid.

Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2020-12-16 19:31:46 +03:00
Pavel Solodovnikov
3a037bc5b6 idl: remove redundant set_namespace function
Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2020-12-16 19:31:40 +03:00
Pavel Solodovnikov
e76e8aec0e idl: remove unused declare_class function
Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2020-12-16 19:31:35 +03:00
Pavel Solodovnikov
745f4ac23b idl: slightly change str and repr for AST types
Surround string representation with angle brackets. This improves
readability when printing debug output.

Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2020-12-16 19:31:20 +03:00
Pavel Solodovnikov
4a61270701 idl: place directly executed init code into if __name__=="__main__"
Since idl compiler is not intended to be used as a module to other
python build scripts, move initialization code under an if checking
that current module name is "__main__".

Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2020-12-16 19:30:33 +03:00
Pavel Solodovnikov
8b8dce15c3 idl: add docstrings for AST classes
Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2020-12-16 09:03:39 +03:00
Pavel Solodovnikov
facf27dbe4 idl: allow to parse const specifiers for template arguments
This patch introduces very limited support for declaring `const`
template parameters in data members.

It's not covering all the cases, e.g.
`const type member_variable` and `const template_def<T1, T2, ...>`
syntax is not supported at the moment.

Though the changes are enough for raft-related use: this makes it
possible to declare `std::vector<raft::log_entries_ptr>` (aka
`std::vector<lw_shared_ptr<const raft::log_entry>>`) in the IDL.

Existing IDL files are not affected in any way.

Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2020-12-15 16:03:11 +03:00
Pavel Solodovnikov
f02703fcd7 idl: fix a few typos in idl-compiler
Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2020-12-15 16:02:55 +03:00
Pavel Solodovnikov
28b602833f idl: switch from string.Template to python f-strings and format string in idl-compiler
Move to a modern and lightweight syntax of f-strings
introduced in python 3.6. It improves readability and provides
greater flexibility.

A few places are now using format strings instead, though.

In case when multiline substitution variable is used, the template
string should be first re-indented and only after that the
formatting should be applied, or we can end up with screwed
indentation the in generated sources.

This change introduces one invisible whitespace change
in `query.dist.impl.hh`, otherwise all generated code is exactly
the same.

Tests: build(dev) and diff genetated IDL sources by hand

Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2020-12-15 16:01:17 +03:00
Pavel Solodovnikov
4ab1f7f55d idl: Decouple idl-compiler data structures from grammar structure
Instead of operating on the raw lists of tokens, transform them into
typed structures representation, which makes the code by many orders of
magnitude simpler to read, understand and extend.

This includes sweeping changes throughout the whole source code of the
tool, because almost every function was tightly coupled to the way
data was passed down from the parser right to the code generation
routines.

Tested manually by checking that old generated sources are precisely
the same as the new generated sources.

Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2020-12-15 15:59:17 +03:00
Avi Kivity
a99aba9e48 idl-compiler: generate views after serializers
Clang eagerly instantiates templates, so if it needs a template
function for which it has a declaration but not a definition, it
will not instantiate the definition when it sees it. This causes
link errors.

In this case, the views use the serializer implementations, but are
generated before them.

Fix by generating the view implementations after the serializer
implementations that they use.
2020-10-03 19:56:25 +03:00
Avi Kivity
a4c44cab88 treewide: update concepts language from the Concepts TS to C++20
Seastar recently lost support for the experimental Concepts Technical
Specification (TS) and gained support for C++20 concepts. Re-enable
concepts in Scylla by updating our use of concepts to the C++20
standard.

This change:
 - peels off uses of the GCC6_CONCEPT macro
 - removes inclusions of <seastar/gcc6-concepts.hh>
 - replaces function-style concepts (no longer supported) with
   equation-style concepts
 - semicolons added and removed as needed
 - deprecated std::is_pod replaced by recommended replacement
 - updates return type constraints to use concepts instead of
   type names (either std::same_as or std::convertible_to, with
   std::same_as chosen when possible)

No attempt is made to improve the concepts; this is a specification
update only.
Message-Id: <20200531110254.2555854-1-avi@scylladb.com>
2020-06-02 09:12:21 +03:00
Botond Dénes
838b92f4b0 idl-compiler.py: don't use 'is not' for string comparison
In python, `is` and `is not` checks object identity, not value
equivalence, yet in `idl-compiler.py` it is used to compare strings.
Newer python versions (that shipped in Fedora32) complains about this
misuse, so this patch fixes it.

Tests: unit(dev)
Signed-off-by: Botond Dénes <bdenes@scylladb.com>
Message-Id: <20200526091811.50229-1-bdenes@scylladb.com>
2020-05-27 08:40:05 +03:00
Rafael Ávila de Espíndola
72e900291b build: Make the output of idl-compiler deterministic
If at any point during the topological sort we had more than one node
with zero dependencies, the order they were printed was not
deterministic.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2020-01-22 16:28:00 -08:00
Duarte Nunes
fa2b0384d2 Replace std::experimental types with C++17 std version.
Replace stdx::optional and stdx::string_view with the C++ std
counterparts.

Some instances of boost::variant were also replaced with std::variant,
namely those that called seastar::visit.

Scylla now requires GCC 8 to compile.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20190108111141.5369-1-duarte@scylladb.com>
2019-01-08 13:16:36 +02:00
Alexys Jacob
14e65e1089 idl-compiler.py: python3 shebang 2018-11-28 23:58:38 +01:00
Alexys Jacob
e58eb6d6ab idl-compiler.py: coding style fixes
idl-compiler.py:22:1: F401 'json' imported but unused
idl-compiler.py:23:1: F401 'sys' imported but unused
idl-compiler.py:24:1: F401 're' imported but unused
idl-compiler.py:25:1: F401 'glob' imported but unused
idl-compiler.py:27:1: F401 'os' imported but unused
idl-compiler.py:54:1: F811 redefinition of unused 'reindent' from line 33
idl-compiler.py:57:1: E302 expected 2 blank lines, found 1
idl-compiler.py:61:1: E302 expected 2 blank lines, found 1
idl-compiler.py:66:1: E302 expected 2 blank lines, found 1
idl-compiler.py:96:1: E302 expected 2 blank lines, found 1
idl-compiler.py:160:1: E302 expected 2 blank lines, found 1
idl-compiler.py:163:1: E302 expected 2 blank lines, found 1
idl-compiler.py:166:1: E302 expected 2 blank lines, found 1
idl-compiler.py:172:1: E302 expected 2 blank lines, found 1
idl-compiler.py:176:1: E302 expected 2 blank lines, found 1
idl-compiler.py:176:47: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:176:49: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:191:24: E203 whitespace before ':'
idl-compiler.py:191:43: E203 whitespace before ':'
idl-compiler.py:191:67: E203 whitespace before ':'
idl-compiler.py:191:84: E202 whitespace before '}'
idl-compiler.py:195:1: E302 expected 2 blank lines, found 1
idl-compiler.py:195:45: E203 whitespace before ','
idl-compiler.py:195:69: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:195:71: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:198:28: E225 missing whitespace around operator
idl-compiler.py:198:40: E225 missing whitespace around operator
idl-compiler.py:198:43: E272 multiple spaces before keyword
idl-compiler.py:212:25: E203 whitespace before ':'
idl-compiler.py:212:45: E203 whitespace before ':'
idl-compiler.py:212:100: E203 whitespace before ':'
idl-compiler.py:218:1: E302 expected 2 blank lines, found 1
idl-compiler.py:225:1: E302 expected 2 blank lines, found 1
idl-compiler.py:226:11: E271 multiple spaces after keyword
idl-compiler.py:228:1: E302 expected 2 blank lines, found 1
idl-compiler.py:235:1: E302 expected 2 blank lines, found 1
idl-compiler.py:238:1: E302 expected 2 blank lines, found 1
idl-compiler.py:241:5: E722 do not use bare 'except'
idl-compiler.py:243:1: E305 expected 2 blank lines after class or function
definition, found 0
idl-compiler.py:245:1: E302 expected 2 blank lines, found 1
idl-compiler.py:250:25: E231 missing whitespace after ','
idl-compiler.py:253:1: E302 expected 2 blank lines, found 1
idl-compiler.py:256:1: E302 expected 2 blank lines, found 1
idl-compiler.py:263:1: E302 expected 2 blank lines, found 1
idl-compiler.py:266:1: E302 expected 2 blank lines, found 1
idl-compiler.py:267:75: E225 missing whitespace around operator
idl-compiler.py:269:1: E302 expected 2 blank lines, found 1
idl-compiler.py:272:1: E302 expected 2 blank lines, found 1
idl-compiler.py:275:1: E302 expected 2 blank lines, found 1
idl-compiler.py:278:1: E305 expected 2 blank lines after class or function
definition, found 1
idl-compiler.py:280:1: E302 expected 2 blank lines, found 1
idl-compiler.py:283:1: E302 expected 2 blank lines, found 1
idl-compiler.py:286:1: E302 expected 2 blank lines, found 1
idl-compiler.py:288:1: E302 expected 2 blank lines, found 0
idl-compiler.py:293:1: E302 expected 2 blank lines, found 1
idl-compiler.py:294:20: E203 whitespace before ':'
idl-compiler.py:294:22: E241 multiple spaces after ':'
idl-compiler.py:294:51: E203 whitespace before ':'
idl-compiler.py:294:55: E202 whitespace before '}'
idl-compiler.py:296:1: E302 expected 2 blank lines, found 1
idl-compiler.py:298:23: E203 whitespace before ':'
idl-compiler.py:300:1: E305 expected 2 blank lines after class or function
definition, found 1
idl-compiler.py:301:1: E302 expected 2 blank lines, found 0
idl-compiler.py:304:1: E302 expected 2 blank lines, found 1
idl-compiler.py:304:45: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:304:47: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:311:67: E202 whitespace before '}'
idl-compiler.py:314:74: E241 multiple spaces after ':'
idl-compiler.py:316:114: E241 multiple spaces after ':'
idl-compiler.py:316:129: E203 whitespace before ':'
idl-compiler.py:326:1: E302 expected 2 blank lines, found 1
idl-compiler.py:328:27: E231 missing whitespace after ','
idl-compiler.py:328:34: E225 missing whitespace around operator
idl-compiler.py:330:1: E302 expected 2 blank lines, found 1
idl-compiler.py:332:5: F841 local variable 'typ' is assigned to but never used
idl-compiler.py:348:63: E202 whitespace before '}'
idl-compiler.py:352:1: E302 expected 2 blank lines, found 1
idl-compiler.py:353:21: E231 missing whitespace after ','
idl-compiler.py:368:30: E203 whitespace before ':'
idl-compiler.py:374:30: E203 whitespace before ':'
idl-compiler.py:411:57: E203 whitespace before ':'
idl-compiler.py:413:1: E302 expected 2 blank lines, found 1
idl-compiler.py:413:64: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:413:66: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:413:80: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:413:82: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:413:98: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:413:100: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:415:51: E225 missing whitespace around operator
idl-compiler.py:417:57: E225 missing whitespace around operator
idl-compiler.py:448:1: E302 expected 2 blank lines, found 1
idl-compiler.py:448:60: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:448:62: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:448:76: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:448:78: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:448:94: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:448:96: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:451:51: E225 missing whitespace around operator
idl-compiler.py:453:57: E225 missing whitespace around operator
idl-compiler.py:455:30: E231 missing whitespace after ','
idl-compiler.py:477:1: E302 expected 2 blank lines, found 1
idl-compiler.py:477:48: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:477:50: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:477:67: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:477:69: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:484:24: E222 multiple spaces after operator
idl-compiler.py:488:74: E203 whitespace before ':'
idl-compiler.py:498:20: E222 multiple spaces after operator
idl-compiler.py:507:68: E203 whitespace before ':'
idl-compiler.py:507:88: E203 whitespace before ':'
idl-compiler.py:514:87: E231 missing whitespace after ','
idl-compiler.py:520:14: E211 whitespace before '('
idl-compiler.py:521:15: E703 statement ends with a semicolon
idl-compiler.py:523:1: E302 expected 2 blank lines, found 1
idl-compiler.py:540:47: E231 missing whitespace after ':'
idl-compiler.py:542:1: E302 expected 2 blank lines, found 1
idl-compiler.py:542:47: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:542:49: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:542:69: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:542:71: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:547:24: E222 multiple spaces after operator
idl-compiler.py:553:47: E231 missing whitespace after ':'
idl-compiler.py:558:43: E231 missing whitespace after ':'
idl-compiler.py:560:1: E302 expected 2 blank lines, found 1
idl-compiler.py:564:1: E302 expected 2 blank lines, found 1
idl-compiler.py:564:82: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:564:84: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:564:105: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:564:107: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:573:21: E222 multiple spaces after operator
idl-compiler.py:576:25: E222 multiple spaces after operator
idl-compiler.py:577:13: F841 local variable 'sate' is assigned to but never
used
idl-compiler.py:584:66: E203 whitespace before ':'
idl-compiler.py:589:66: E203 whitespace before ':'
idl-compiler.py:589:89: E203 whitespace before ':'
idl-compiler.py:589:113: E203 whitespace before ':'
idl-compiler.py:600:48: E203 whitespace before ':'
idl-compiler.py:600:68: E203 whitespace before ':'
idl-compiler.py:602:1: E302 expected 2 blank lines, found 1
idl-compiler.py:602:1: F811 redefinition of unused 'add_vector_node' from line
330
idl-compiler.py:604:38: E231 missing whitespace after ','
idl-compiler.py:604:59: E202 whitespace before ')'
idl-compiler.py:607:1: E305 expected 2 blank lines after class or function
definition, found 1
idl-compiler.py:609:1: E302 expected 2 blank lines, found 1
idl-compiler.py:615:39: E231 missing whitespace after ','
idl-compiler.py:622:1: E302 expected 2 blank lines, found 1
idl-compiler.py:630:46: E203 whitespace before ':'
idl-compiler.py:637:33: E231 missing whitespace after ':'
idl-compiler.py:640:90: E203 whitespace before ':'
idl-compiler.py:641:13: F841 local variable 'vr' is assigned to but never used
idl-compiler.py:642:1: E305 expected 2 blank lines after class or function
definition, found 0
idl-compiler.py:644:1: E302 expected 2 blank lines, found 1
idl-compiler.py:657:1: E302 expected 2 blank lines, found 1
idl-compiler.py:657:51: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:657:53: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:657:67: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:657:69: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:660:5: E265 block comment should start with '# '
idl-compiler.py:679:16: E272 multiple spaces before keyword
idl-compiler.py:692:56: E271 multiple spaces after keyword
idl-compiler.py:695:5: F841 local variable 'is_param_vector' is assigned to
but never used
idl-compiler.py:699:1: E302 expected 2 blank lines, found 1
idl-compiler.py:699:56: E202 whitespace before ')'
idl-compiler.py:711:1: E302 expected 2 blank lines, found 1
idl-compiler.py:719:26: E201 whitespace after '{'
idl-compiler.py:730:39: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:730:41: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:733:1: E302 expected 2 blank lines, found 1
idl-compiler.py:735:21: E225 missing whitespace around operator
idl-compiler.py:738:1: E302 expected 2 blank lines, found 1
idl-compiler.py:747:1: E305 expected 2 blank lines after class or function
definition, found 1
idl-compiler.py:749:1: E302 expected 2 blank lines, found 1
idl-compiler.py:767:17: E211 whitespace before '('
idl-compiler.py:767:26: E203 whitespace before ':'
idl-compiler.py:770:5: E303 too many blank lines (2)
idl-compiler.py:777:20: E211 whitespace before '('
idl-compiler.py:777:29: E203 whitespace before ':'
idl-compiler.py:783:28: E203 whitespace before ':'
idl-compiler.py:783:44: E203 whitespace before ':'
idl-compiler.py:783:82: E203 whitespace before ':'
idl-compiler.py:786:1: E302 expected 2 blank lines, found 1
idl-compiler.py:794:28: E203 whitespace before ':'
idl-compiler.py:802:33: E203 whitespace before ':'
idl-compiler.py:815:21: E126 continuation line over-indented for hanging
indent
idl-compiler.py:815:28: E203 whitespace before ':'
idl-compiler.py:815:50: E203 whitespace before ':'
idl-compiler.py:817:82: E203 whitespace before ':'
idl-compiler.py:817:104: E203 whitespace before ':'
idl-compiler.py:827:33: E203 whitespace before ':'
idl-compiler.py:827:48: E203 whitespace before ':'
idl-compiler.py:827:68: E203 whitespace before ':'
idl-compiler.py:827:84: E203 whitespace before ':'
idl-compiler.py:827:100: E203 whitespace before ':'
idl-compiler.py:859:24: E203 whitespace before ':'
idl-compiler.py:859:58: E203 whitespace before ':'
idl-compiler.py:859:78: E203 whitespace before ':'
idl-compiler.py:861:1: E302 expected 2 blank lines, found 1
idl-compiler.py:865:1: E302 expected 2 blank lines, found 1
idl-compiler.py:876:1: E302 expected 2 blank lines, found 1
idl-compiler.py:876:71: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:876:73: E251 unexpected spaces around keyword / parameter
equals
idl-compiler.py:883:21: E222 multiple spaces after operator
idl-compiler.py:884:28: E225 missing whitespace around operator
idl-compiler.py:884:46: E225 missing whitespace around operator
idl-compiler.py:884:49: E272 multiple spaces before keyword
idl-compiler.py:904:86: E203 whitespace before ':'
idl-compiler.py:904:107: E203 whitespace before ':'
idl-compiler.py:906:81: E203 whitespace before ':'
idl-compiler.py:906:106: E203 whitespace before ':'
idl-compiler.py:906:124: E203 whitespace before ':'
idl-compiler.py:906:143: E203 whitespace before ':'
idl-compiler.py:911:49: E203 whitespace before ':'
idl-compiler.py:911:69: E203 whitespace before ':'
idl-compiler.py:911:93: E203 whitespace before ':'
idl-compiler.py:918:85: E203 whitespace before ':'
idl-compiler.py:918:108: E203 whitespace before ':'
idl-compiler.py:918:151: E203 whitespace before ':'
idl-compiler.py:922:62: E203 whitespace before ':'
idl-compiler.py:922:90: E203 whitespace before ':'
idl-compiler.py:925:82: E203 whitespace before ':'
idl-compiler.py:925:110: E203 whitespace before ':'
idl-compiler.py:940:70: E203 whitespace before ':'
idl-compiler.py:940:128: E203 whitespace before ':'
idl-compiler.py:942:110: E203 whitespace before ':'
idl-compiler.py:942:168: E203 whitespace before ':'
idl-compiler.py:948:25: E203 whitespace before ':'
idl-compiler.py:948:75: E203 whitespace before ':'
idl-compiler.py:954:78: E203 whitespace before ':'
idl-compiler.py:954:101: E203 whitespace before ':'
idl-compiler.py:954:144: E203 whitespace before ':'
idl-compiler.py:957:62: E203 whitespace before ':'
idl-compiler.py:957:90: E203 whitespace before ':'
idl-compiler.py:969:13: E271 multiple spaces after keyword
idl-compiler.py:971:13: E271 multiple spaces after keyword
idl-compiler.py:976:1: E302 expected 2 blank lines, found 1
idl-compiler.py:987:1: E302 expected 2 blank lines, found 1
idl-compiler.py:1016:1: E302 expected 2 blank lines, found 1
idl-compiler.py:1023:42: E225 missing whitespace around operator
idl-compiler.py:1024:79: E225 missing whitespace around operator
idl-compiler.py:1027:1: E305 expected 2 blank lines after class or function
definition, found 0

Signed-off-by: Alexys Jacob <ultrabug@gentoo.org>
Message-Id: <20181104112308.19409-1-ultrabug@gentoo.org>
2018-11-14 19:25:13 +02:00
Paweł Dziepak
3b7579aa0e idl-compiler: specify return type of with_serialized_stream() lambdas
IDL-generated code uses with_serialized_stream() to optimise for cases
when the underlying buffer is not fragmented. The provided lambda will
be called with wither simple or fragmented stream as an argument. The
consequence of this is that both instantations of generic lambda need to
return the same type. This is a problem if the type is deduced and
depends on the provided input stream (e.g. different type for fragmented
and simple streams). The solution is to explictly specify the return
type as the type returned by deserialising general utils::input_stream.
This way each instantation of lambda can return whatever it wants as
long as it is convertible to the type that the serialiser would return
if utils::input_stream was given.
2018-08-24 16:07:20 +01:00
Piotr Sarna
450e014558 idl: remove for_each from fragmented serialization
Previously fragmented buffers of bytes were serialized
with a for_each loop. Since serializing bytes involves writing
size first and then data, only first fragment (and its size)
would be taken into account.
This commit changes fragmented code generation so it expects
that serialized range has a serialize(output, T) specification
and expects it to iterate over fragments on its own (just like
serializer for basic_value_view does).

Fixes #3501
2018-06-13 13:54:09 +02:00
Paweł Dziepak
5845d52632 idl: allow fragmented bytes_view in serialisation
This patch adds new way of serialising bytes and sstring objects in the
IDL. Using write_fragmented_<field-name>() the caller can pass a range
of fragments that would be serialised without linearising the buffer.
2018-05-31 10:09:01 +01:00
Paweł Dziepak
a5731ded98 idl: create variant state in root node
Each non-final IDL object is preceeded by a frame containing its size.
In case of boost::variant there is a frame for the variant itself, an
integer determining the active alternative of the variant and a frame of
that active alternative.

However, if a variant was the first member of a writable stub object the
IDL would generate code that would not write the frame for the variant.
This is not a very severe issue since there are no such cases right now
as  C++ type system would no allow such generated code to compile.
2018-05-25 10:15:10 +01:00
Paweł Dziepak
f719516be8 idl-compiler: avoid unused variable in empty struct deserialisers
Deserialisers generated by IDL compiler first create a substream
covering the deserialised structure and then skip and read appropriate
members. If there are no members the substream will be unused and prompt
the compiler to emit a warning.
2018-05-25 10:15:10 +01:00
Duarte Nunes
8cc29f84fb idl-compiler: Support optional fields in views
When generating view code, the compiler was ignoring optional fields.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-04-25 11:43:04 +02:00
Paweł Dziepak
4df4994b71 idl: fix generated writers when member functions are used
When using member name in an idetifer of generated class or method
idl compiler should strip the trailing '()'.
2017-02-27 17:05:58 +00:00
Paweł Dziepak
38ee69dee0 idl: allow writers to use any output stream
Original IDL generated code was hardcoded to always use bytes_ostream.
This patch makes the output stream a template parameter so that any
valid output stream can be used.
Unfortunately, making IDL writers generic requires updates in the code
that uses them, this is fixed in C++17 which would be able to deduce the
parameter in most cases.
2016-12-22 13:35:04 +01:00
Gleb Natapov
2e8b255741 Merge seastar upstream
* seastar 0303e0c...e534401 (6):
  > Merge "enable rpc to work on non contiguous memory for receive" from Gleb
  > install-dependencies.sh: install python3 for Ubuntu/Debian, which requires for configure.py
  > fix tcp stuck when output_stream write more than 212992 bytes once.
  > scripts/posix_net_conf.sh: supress 'ls: cannot access /sys/class/net/<NIC>/device/msi_irqs/' error message
  > scripts/posix_net_conf.sh: fix 'command not found' error when specifies --cpu-mask
  > native_network_stack: Fix use after free/missing wait in dhcp

Includes: "Remove utils::fragmented_input_stream and utils::input_stream in favor of seastar version" from Gleb.
2016-09-15 12:12:16 +03:00
Paweł Dziepak
89f7b46f61 idl: switch to utils::input_stream
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
dcf794b04d idl: make bytes compatible with bytes_ostream
This patch makes idl type "bytes" compatible with both bytes and
bytes_ostream.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Tomasz Grabiec
c3707ff754 idl: Avoid full deserialization in skip()
Fixes #1330.
2016-07-25 17:35:34 +02:00
Tomasz Grabiec
207c8d94f1 idl: Rename variable to a more meaningful name
Message-Id: <1465909911-10534-2-git-send-email-tgrabiec@scylladb.com>
2016-06-14 17:02:59 +03:00
Paul McGuire
8326fe7760 Clean up idl-compiler pyparsing usage
This patch makes a few minor improvements in the parser:

  - merge first and rest into 2-argument form of Word to define
    identifier – should give some performance boost, simpler code

  - replace Literal(keyword_string) with Keyword(keyword_string)
    throughout - stricter parsing, avoids misinterpreting identifiers
    with keywords

  - replace expr.setResultsName("name") with expr("name") throughout –
    this is a style change (no actual change in underlying parser
    behavior), but I find this form easier to follow

  - add calls to setName to make exceptions more readable

Message-Id: <005901d1bbd2$711f7bb0$535e7310$@austin.rr.com>
2016-06-08 08:13:05 +03:00
Duarte Nunes
9bd7d08fc7 idl-compiler: Default expr can refer to previous fields
This patch changes the idl-compiler so that the default value of a
field can be set to the value of a previous field in the class:

class P {
    uint32_t x;
    uint32_t y = x;
};

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-06-02 16:21:36 +02:00
Amnon Heiman
e26002d581 idl-compiler: default constructor of complex types
This patch solve a problem where a complex type is define as version
depended (with the version attribute) but doesn't have a default value.

In those cases the default constructor is used, but in the case of
complex types (template) param_type should be use to get the C++ type.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1463916723-15322-1-git-send-email-amnon@scylladb.com>
2016-05-22 15:32:29 +03:00
Tomasz Grabiec
d0ae2e3607 idl-compiler: Make sub-streams stored in views be properly bounded
Currently when reading a view to an object the stream stored has the
same bound as the containing stream, not the bounds of the object
itself. The serializer of the view assumes that the stream has the
bounds of the object itself.

Fixes dtest failure in
paging_test.py:TestPagingSize.test_undefined_page_size_default

Fixes #963.

Message-Id: <1456854556-32088-1-git-send-email-tgrabiec@scylladb.com>
2016-03-01 19:50:42 +02:00
Amnon Heiman
1c7bc28d35 idl-compiler: change optional vector implementation
This patch change the way optional vector are implemented.

Now a vector of optional would be handle like any other non primitive
types, with a single method add() that would return a writer to the
optional.

The writer to the optional would have a skip and write method like
simple optional field.

For basic types the write method would get the value as a parameter, for
composite type, it would return a writer to the type.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1456796143-3366-2-git-send-email-amnon@scylladb.com>
2016-03-01 09:41:30 +02:00
Tomasz Grabiec
ee8509cf36 idl-compiler: Introduce add(*_view) on vector 2016-02-26 12:26:13 +01:00
Tomasz Grabiec
4ab0ca07f1 idl-compiler: Catch un-closed frame errors sooner
By initilizing them to 0 we can catch unclosed frames at
deserialization time. It's better than leaving frame size undefined,
which may cause errors much later in deserialization process and thus
would make it harder to identifiy the real cause.
2016-02-26 12:26:13 +01:00
Amnon Heiman
9ea3ffe527 idl-compiler: Add optional support
This patch adds optional writer support an optional field can be either
skip or set.

For vector of optional, a write_empty method will
add 1 to the vector count and mark the optional as false.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-02-26 12:25:08 +01:00
Tomasz Grabiec
97558b2cfe idl-compiler: Put serializers inside template class specializations
The problem is that a generic functions (eg. skip()) which call
deserialize() overloads based on their template parameter only see
deserilize() overloads which were declared at the time skip() was
declared and not those which are available at the time of
instantiation. This forces all serializers to be declared before
serialization_visitors.hh is first included. Serializers included
later will fail to compile. This becomes problematic to ensure when
serializers are included from headers.

Template class specialization lookup doesn't suffer from this
limitation. We can use that to solve the problem. The IDL compiler
will now generate template class specializations with read/write
static methods. In addition to that, default serializer() and
deserialize() implementations are delegating to serializer<>
specialization so that API and existing code doesn't have to change.

Message-Id: <1456423066-6979-1-git-send-email-tgrabiec@scylladb.com>
2016-02-25 20:00:49 +02:00
Amnon Heiman
f461ebc411 idl-compiler: Add pos and rollback to serialize vector
This adds the ability to store a position of a serialized vector and to
rollback to that stored position afterwards.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1456041750-1505-3-git-send-email-amnon@scylladb.com>
2016-02-23 17:49:51 +01:00
Paweł Dziepak
28fa2a6493 idl-compiler: add serialization callback interface
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:50:29 +00:00
Paweł Dziepak
7586e47004 idl-compiler: avoid copy of basic types
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:47:42 +00:00
Paweł Dziepak
2ea735f5ed idl-compiler: accept both bytes and bytes_view
bytes can always be trivially converted to bytes_view. Conversion in the
other direction requires a copy.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:47:42 +00:00