Question about IntrusivePtr and incomplete types

Development
1

Hi,

I just tried to use our new IntrusivePtr type for the first time - and encountered a (for me) unexpected problem.

In my specific case, I want to introduce a new member variable for TableVal. The type of it is:

   IntrusivePtr<Expr> change_func;

Compiling Zeek with this addition to the TableVal class will yield in the following compilation error:

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DDOCTEST_CONFIG_DISABLE -Isrc/zeekygen -I../src/zeekygen -Isrc -I../src -I/opt/local/include -Iaux/binpac/lib -I../aux/binpac/lib -I../aux/paraglob/include -I../aux/broker/3rdparty/caf/libcaf_test -I../aux/broker/3rdparty/caf/libcaf_openssl -I../aux/broker/3rdparty/caf/libcaf_io -Iaux/broker/caf-build/libcaf_core -I../aux/broker/3rdparty/caf/libcaf_core -Iaux/broker/include -I../aux/broker/include -I. -Wall -Wno-unused -g -DDEBUG -DBRO_DEBUG -Wno-register -std=c++17 -g -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -mmacosx-version-min=10.14 -MD -MT src/zeekygen/CMakeFiles/bro_zeekygen.dir/ScriptInfo.cc.o -MF src/zeekygen/CMakeFiles/bro_zeekygen.dir/ScriptInfo.cc.o.d -o src/zeekygen/CMakeFiles/bro_zeekygen.dir/ScriptInfo.cc.o -c ../src/zeekygen/ScriptInfo.cc
In file included from ../src/zeekygen/ScriptInfo.cc:7:
In file included from ../src/zeekygen/Manager.h:14:
In file included from ../src/Val.h:23:
../src/IntrusivePtr.h:88:4: error: no matching function for call to 'Unref'
                         Unref(ptr_);
                         ^~~~~
../src/Val.h:903:2: note: in instantiation of member function 'IntrusivePtr<Expr>::~IntrusivePtr' requested here
         TableVal() {}
         ^
../src/Obj.h:199:13: note: candidate function not viable: cannot convert argument of incomplete type 'IntrusivePtr<Expr>::pointer' (aka 'Expr *') to 'BroObj *' for 1st argument
inline void Unref(BroObj* o)
             ^
Including Expr.h in Val.h is not trivially possible due to other ordering conflicts breaking the build.

Is there some way to use an IntrusivePtr in this case - or is is just not possible to use it in cases where it is not possible to use complete types?

Thanks,
  Johanna

2

I just tried to use our new IntrusivePtr type for the first time - and
encountered a (for me) unexpected problem.

In my specific case, I want to introduce a new member variable for
TableVal. The type of it is:

  IntrusivePtr<Expr> change_func;

Compiling Zeek with this addition to the TableVal class will yield in
the following compilation error:

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-DDOCTEST_CONFIG_DISABLE -Isrc/zeekygen -I../src/zeekygen -Isrc
-I../src -I/opt/local/include -Iaux/binpac/lib -I../aux/binpac/lib
-I../aux/paraglob/include -I../aux/broker/3rdparty/caf/libcaf_test
-I../aux/broker/3rdparty/caf/libcaf_openssl
-I../aux/broker/3rdparty/caf/libcaf_io
-Iaux/broker/caf-build/libcaf_core
-I../aux/broker/3rdparty/caf/libcaf_core -Iaux/broker/include
-I../aux/broker/include -I. -Wall -Wno-unused -g -DDEBUG -DBRO_DEBUG
-Wno-register -std=c++17 -g -isysroot
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk
-mmacosx-version-min=10.14 -MD -MT
src/zeekygen/CMakeFiles/bro_zeekygen.dir/ScriptInfo.cc.o -MF
src/zeekygen/CMakeFiles/bro_zeekygen.dir/ScriptInfo.cc.o.d -o
src/zeekygen/CMakeFiles/bro_zeekygen.dir/ScriptInfo.cc.o -c
../src/zeekygen/ScriptInfo.cc
In file included from ../src/zeekygen/ScriptInfo.cc:7:
In file included from ../src/zeekygen/Manager.h:14:
In file included from ../src/Val.h:23:
../src/IntrusivePtr.h:88:4: error: no matching function for call to
'Unref'
                        Unref(ptr_);
                        ^~~~~
../src/Val.h:903:2: note: in instantiation of member function
'IntrusivePtr<Expr>::~IntrusivePtr' requested here
        TableVal() {}
        ^
../src/Obj.h:199:13: note: candidate function not viable: cannot convert
argument of incomplete type 'IntrusivePtr<Expr>::pointer' (aka 'Expr *')
to 'BroObj *' for 1st argument
inline void Unref(BroObj* o)
            ^
Including Expr.h in Val.h is not trivially possible due to other
ordering conflicts breaking the build.

Is there some way to use an IntrusivePtr in this case - or is is just
not possible to use it in cases where it is not possible to use complete
types?

You should be able to get away with forward declarations as long as you provide matching prototypes for Ref and Unref.

3

Is there some way to use an IntrusivePtr in this case - or is is just
not possible to use it in cases where it is not possible to use complete
types?

You should be able to get away with forward declarations as long as you provide matching prototypes for Ref and Unref.

If I do that I get linking errors:

   "Ref(Expr*)", referenced from:
       Expr::Ref() in IdentifierInfo.cc.o
       Expr::Ref() in Expr.cc.o
       Expr::Ref() in ID.cc.o
       WhenStmt::Exec(Frame*, stmt_flow_type&) const in Stmt.cc.o
       Expr::Ref() in Type.cc.o
       Expr::Ref() in Val.cc.o
       make_var(ID*, BroType*, init_class, Expr*, List<Attr*>*, decl_type, int) in Var.cc.o
       ...
   "Unref(Expr*)", referenced from:
       zeekygen::IdentifierInfo::Redefinition::~Redefinition() in IdentifierInfo.cc.o
       Attr::~Attr() in Attr.cc.o
       UnaryExpr::~UnaryExpr() in Expr.cc.o
       BinaryExpr::~BinaryExpr() in Expr.cc.o
       CondExpr::~CondExpr() in Expr.cc.o
       ScheduleExpr::~ScheduleExpr() in Expr.cc.o
       CallExpr::~CallExpr() in Expr.cc.o
       ...

Which makes sense, because Ref/Unref(Expr*) never exist - they only exist for BroObj*. I could implement a forward function that does nothing besides calling Ref(BroObj) in Expr.cc - but that seems… not really elegant/worth using anymore in this case. Unless I an declare them in another way that I am missing.

Johanna

4

For completeness - the forward declaration/prototype in Val.h is:

class Expr;
void Ref(Expr*);
void Unref(Expr*);

Johanna

5

Is there some way to use an IntrusivePtr in this case - or is is just
not possible to use it in cases where it is not possible to use complete
types?

You should be able to get away with forward declarations as long as you provide matching prototypes for Ref and Unref.

If I do that I get linking errors:

"Ref(Expr*)", referenced from:
     Expr::Ref() in IdentifierInfo.cc.o
     Expr::Ref() in Expr.cc.o
     Expr::Ref() in ID.cc.o
     WhenStmt::Exec(Frame*, stmt_flow_type&) const in Stmt.cc.o
     Expr::Ref() in Type.cc.o
     Expr::Ref() in Val.cc.o
     make_var(ID*, BroType*, init_class, Expr*, List<Attr*>*, decl_type, int) in Var.cc.o
     ...
"Unref(Expr*)", referenced from:
     zeekygen::IdentifierInfo::Redefinition::~Redefinition() in IdentifierInfo.cc.o
     Attr::~Attr() in Attr.cc.o
     UnaryExpr::~UnaryExpr() in Expr.cc.o
     BinaryExpr::~BinaryExpr() in Expr.cc.o
     CondExpr::~CondExpr() in Expr.cc.o
     ScheduleExpr::~ScheduleExpr() in Expr.cc.o
     CallExpr::~CallExpr() in Expr.cc.o
     ...

Which makes sense, because Ref/Unref(Expr*) never exist - they only exist for BroObj*. I could implement a forward function that does nothing besides calling Ref(BroObj) in Expr.cc - but that seems… not really elegant/worth using anymore in this case. Unless I an declare them in another way that I am missing.

I agree, that extra bit of boilerplate is not very elegant. However, the compiler has no knowledge of the class hierarchy of Expr and thus doesn’t know that Expr* is convertible to BroObj*. Hence, we need the one extra layer of indirection via Ref/Unref overloads for Expr.

Implementing the functions like this should make the linker happy as well:

void Ref(Expr* ptr) { Ref(static_cast<BroObj*>(ptr); }
void Unref(Expr* ptr) { Unref(static_cast<BroObj*>(ptr); }

There is one alternative: The compiler only needs to see Ref/Unref overloads when *instantiating* any member function of IntrusivePtr. You probably have some inline constructor or destructor that causes the compiler to instantiate the template. Moving *all* member function that operate on the IntrusivePtr member out of the .h file should work as well.

For example:

struct foo;

struct foobar {
  foobar();
  ~foobar();
  IntrusivePtr<foo> f;
};

should compile, whereas

struct foo;

struct foobar {
  foobar();
  ~foobar() {}
  IntrusivePtr<foo> f;
};

won’t. In the latter case, providing the destructor inline causes the compiler to instantiate the destructor of `IntrusivePtr<foo>`, which relies on an Unref function it can’t find at this point.