Hi,
(Sent to Einstein Toolkit mailing list because Cactus mailing lists are down)
I am trying to introduce an optional dependency on HDF5 in my thorn. The idea is that the thorn could work either with or without HDF5 support in the configuration, and the code in the thorn would be conditionally compiled depending on whether HDF5 is present.
The Cactus capability mechanism has an "optional" keyword, so you can say that a thorn depends optionally on a particular capability (in this case, HDF5). However, in the Einstein Toolkit thorns, this is very rarely used, and never for conditional compilation, so I suspect that this feature has not been well tested.
Indeed, it appears that it doesn't work. In order to detect at compile time whether a given capability is present, it is necessary to have a preprocessor (#define) macro defined. The file that would do this is Cactus/lib/sbin/CreateConfigurationBindings.pl. This file sets a make-system definition for the given capability (i.e. HDF5 = 1) but it does not set a preprocessor macro. This could be done manually in the configuration script of the thorn providing the capability, by outputting the lines
echo "BEGIN DEFINE" echo "HAVE_HDF5 1" echo "END DEFINE"
This would cause the line "#define HAVE_HDF5 1" to be included whenever the thorn that requires the capability includes cctki_Capabilities.h. However, since this is a "cctki" file (with 'i' for internal?), I assume that it is not supposed to be included by user thorns. Instead, I think it is supposed to be included automatically, but I can't find any code that would do that, and I think it is missing.
Further, there is a line in CreateConfigurationBindings.pl which seems to be wrong. Line 166 says
$incs .= "#define " . $cfg->{"\U$thorn\E OPTIONAL \U$providedcap\E DEFINE"} . " 1\n";
which doesn't make sense because $cfg->{"\U$thorn\E OPTIONAL \U$providedcap\E DEFINE"} is a list of defines, not a single one, and anyway these definitions have already been provided earlier in the file which will be included, so this line is redundant. It leads to the output #define 1, presumably because the list evaluates to an empty string. This causes a compilation failure when you include cctki_Capabilities.h.
1. Do we want all capabilities to generate automatically a preprocessor macro which can be checked? e.g. HAVE_CAPABILITY_HDF5, (to avoid conflicts with autoconf-generated macros)
2. If not, then we could add the BEGIN_DEFINE...END_DEFINE lines above to HDF5.sh to manually set the define.
3. I think line 166 of CreateConfigurationBindings.pl should be removed.
4. There should be some mechanism for each thorn to include cctki_Capabilities.h automatically.
On Mon, Nov 29, 2010 at 5:19 PM, Ian Hinder ian.hinder@aei.mpg.de wrote:
Hi,
(Sent to Einstein Toolkit mailing list because Cactus mailing lists are down)
I am trying to introduce an optional dependency on HDF5 in my thorn. The idea is that the thorn could work either with or without HDF5 support in the configuration, and the code in the thorn would be conditionally compiled depending on whether HDF5 is present.
The Cactus capability mechanism has an "optional" keyword, so you can say that a thorn depends optionally on a particular capability (in this case, HDF5). However, in the Einstein Toolkit thorns, this is very rarely used, and never for conditional compilation, so I suspect that this feature has not been well tested.
Indeed, it appears that it doesn't work. In order to detect at compile time whether a given capability is present, it is necessary to have a preprocessor (#define) macro defined. The file that would do this is Cactus/lib/sbin/CreateConfigurationBindings.pl. This file sets a make-system definition for the given capability (i.e. HDF5 = 1) but it does not set a preprocessor macro. This could be done manually in the configuration script of the thorn providing the capability, by outputting the lines
echo "BEGIN DEFINE" echo "HAVE_HDF5 1" echo "END DEFINE"
This would cause the line "#define HAVE_HDF5 1" to be included whenever the thorn that requires the capability includes cctki_Capabilities.h. However, since this is a "cctki" file (with 'i' for internal?), I assume that it is not supposed to be included by user thorns. Instead, I think it is supposed to be included automatically, but I can't find any code that would do that, and I think it is missing.
Further, there is a line in CreateConfigurationBindings.pl which seems to be wrong. Line 166 says
$incs .= "#define " . $cfg->{"\U$thorn\E OPTIONAL \U$providedcap\E DEFINE"} . " 1\n";
which doesn't make sense because $cfg->{"\U$thorn\E OPTIONAL \U$providedcap\E DEFINE"} is a list of defines, not a single one, and anyway these definitions have already been provided earlier in the file which will be included, so this line is redundant. It leads to the output #define 1, presumably because the list evaluates to an empty string. This causes a compilation failure when you include cctki_Capabilities.h.
- Do we want all capabilities to generate automatically a preprocessor macro which can be checked? e.g. HAVE_CAPABILITY_HDF5, (to avoid conflicts with autoconf-generated macros)
Yes, this should be the case.
- If not, then we could add the BEGIN_DEFINE...END_DEFINE lines above to HDF5.sh to manually set the define.
I don't mind whether it is automatic or whether the scripts provide this.
- I think line 166 of CreateConfigurationBindings.pl should be removed.
Could be -- do you want to post a patch? Or is this the line that needs to be corrected to generate the #defines automatically?
- There should be some mechanism for each thorn to include cctki_Capabilities.h automatically.
Yes, definitely. It should probably be part of cctk.h.
-erik
On 30 Nov 2010, at 02:24, Erik Schnetter wrote:
- Do we want all capabilities to generate automatically a preprocessor macro which can be checked? e.g. HAVE_CAPABILITY_HDF5, (to avoid conflicts with autoconf-generated macros)
Yes, this should be the case.
- If not, then we could add the BEGIN_DEFINE...END_DEFINE lines above to HDF5.sh to manually set the define.
I don't mind whether it is automatic or whether the scripts provide this.
It is easy to add to the script, so I vote for it being automatic.
- I think line 166 of CreateConfigurationBindings.pl should be removed.
Could be -- do you want to post a patch? Or is this the line that needs to be corrected to generate the #defines automatically?
It could be modified to do that, or the required line could be added to the header file of the providing thorn. I think it is more natural to do the latter, since then it would only be added in one place rather than in each thorn that requires the capability.
I attach a patch which:
* Adds a line such as
#define HAVE_CAPABILITY_<cap> 1
where <cap> is the capability name, to the Capabilities/cctki_<cap>.h file. This is the file that is later included by any thorns which require the capability.
* Removes the incorrect line which breaks compilation of cctki_Capabilities.h and does not do what looks like was intended.
- There should be some mechanism for each thorn to include cctki_Capabilities.h automatically.
Yes, definitely. It should probably be part of cctk.h.
I'm not sure about the Cactus conventions for this. There aren't any other cctki_ headers included from cctk.h. The problem with including cctki_Capabilities.h in cctk.h is that when this file is modified, for example when a thorn is added or removed, every source file in the configuration would need to be recompiled. One way around this would be to exclude cctki_Capabilities.h from all thorns' dependencies, as is done with cctk_Arguments.h. However, this would mean that when a new thorn providing a capability was added, the thorns requiring this capability would not be recompiled. My best solution would be to have something like
#include "../Configuration/Thorns/cctki_" CCTK_THORNSTRING ".h"
in cctk.h. However, I can't get this to work. String concatenation happens after preprocessing, and there is no preprocessor operator to concatenate two string literals (the ## operator only works on identifiers, not strings).
The best solution I can get to work is to have all thorns which optionally require capabilities include their corresponding "../Configuration/Thorns/cctki_<thornname>.h" file manually.
On Tue, Nov 30, 2010 at 12:01 PM, Ian Hinder ian.hinder@aei.mpg.de wrote:
On 30 Nov 2010, at 02:24, Erik Schnetter wrote:
- Do we want all capabilities to generate automatically a preprocessor macro which can be checked? e.g. HAVE_CAPABILITY_HDF5, (to avoid conflicts with autoconf-generated macros)
Yes, this should be the case.
- If not, then we could add the BEGIN_DEFINE...END_DEFINE lines above to HDF5.sh to manually set the define.
I don't mind whether it is automatic or whether the scripts provide this.
It is easy to add to the script, so I vote for it being automatic.
- I think line 166 of CreateConfigurationBindings.pl should be removed.
Could be -- do you want to post a patch? Or is this the line that needs to be corrected to generate the #defines automatically?
It could be modified to do that, or the required line could be added to the header file of the providing thorn. I think it is more natural to do the latter, since then it would only be added in one place rather than in each thorn that requires the capability.
I attach a patch which:
- Adds a line such as
#define HAVE_CAPABILITY_<cap> 1
where <cap> is the capability name, to the Capabilities/cctki_<cap>.h file. This is the file that is later included by any thorns which require the capability.
- Removes the incorrect line which breaks compilation of cctki_Capabilities.h and does not do what looks like was intended.
- There should be some mechanism for each thorn to include cctki_Capabilities.h automatically.
Yes, definitely. It should probably be part of cctk.h.
I'm not sure about the Cactus conventions for this. There aren't any other cctki_ headers included from cctk.h. The problem with including cctki_Capabilities.h in cctk.h is that when this file is modified, for example when a thorn is added or removed, every source file in the configuration would need to be recompiled. One way around this would be to exclude cctki_Capabilities.h from all thorns' dependencies, as is done with cctk_Arguments.h. However, this would mean that when a new thorn providing a capability was added, the thorns requiring this capability would not be recompiled. My best solution would be to have something like
#include "../Configuration/Thorns/cctki_" CCTK_THORNSTRING ".h"
in cctk.h. However, I can't get this to work. String concatenation happens after preprocessing, and there is no preprocessor operator to concatenate two string literals (the ## operator only works on identifiers, not strings).
The best solution I can get to work is to have all thorns which optionally require capabilities include their corresponding "../Configuration/Thorns/cctki_<thornname>.h" file manually.
I like the patch; please apply it. Do you want to extend it to also auto-generate a makefile definition HAVE_CAPABILITY_XXX?
I think all we need is to rename cctki_Capabilities.h to cctk_Capabilities.h, and protect it from make the same way cctk_Arguments.h is protected. Then each thorn can include <cctk_Capabilities.h>, and will then automatically include its own capabilities file. Adding/removing thorns won't matter because the file is protected. If a thorn is added that provides a new capability that thorn T uses, then thorn T's file Thorns/cctki_T.h will be modified, and make will recompile the thorn. (This is exactly the same mechanism as for cctk_Arguments.h, and it works fine there.)
-erik
On 30 Nov 2010, at 18:29, Erik Schnetter wrote:
On Tue, Nov 30, 2010 at 12:01 PM, Ian Hinder ian.hinder@aei.mpg.de wrote:
On 30 Nov 2010, at 02:24, Erik Schnetter wrote:
- Do we want all capabilities to generate automatically a preprocessor macro which can be checked? e.g. HAVE_CAPABILITY_HDF5, (to avoid conflicts with autoconf-generated macros)
Yes, this should be the case.
- If not, then we could add the BEGIN_DEFINE...END_DEFINE lines above to HDF5.sh to manually set the define.
I don't mind whether it is automatic or whether the scripts provide this.
It is easy to add to the script, so I vote for it being automatic.
- I think line 166 of CreateConfigurationBindings.pl should be removed.
Could be -- do you want to post a patch? Or is this the line that needs to be corrected to generate the #defines automatically?
It could be modified to do that, or the required line could be added to the header file of the providing thorn. I think it is more natural to do the latter, since then it would only be added in one place rather than in each thorn that requires the capability.
I attach a patch which:
Adds a line such as
#define HAVE_CAPABILITY_<cap> 1where <cap> is the capability name, to the Capabilities/cctki_<cap>.h file. This is the file that is later included by any thorns which require the capability.
- Removes the incorrect line which breaks compilation of cctki_Capabilities.h and does not do what looks like was intended.
- There should be some mechanism for each thorn to include cctki_Capabilities.h automatically.
Yes, definitely. It should probably be part of cctk.h.
I'm not sure about the Cactus conventions for this. There aren't any other cctki_ headers included from cctk.h. The problem with including cctki_Capabilities.h in cctk.h is that when this file is modified, for example when a thorn is added or removed, every source file in the configuration would need to be recompiled. One way around this would be to exclude cctki_Capabilities.h from all thorns' dependencies, as is done with cctk_Arguments.h. However, this would mean that when a new thorn providing a capability was added, the thorns requiring this capability would not be recompiled. My best solution would be to have something like
#include "../Configuration/Thorns/cctki_" CCTK_THORNSTRING ".h"in cctk.h. However, I can't get this to work. String concatenation happens after preprocessing, and there is no preprocessor operator to concatenate two string literals (the ## operator only works on identifiers, not strings).
The best solution I can get to work is to have all thorns which optionally require capabilities include their corresponding "../Configuration/Thorns/cctki_<thornname>.h" file manually.
I like the patch; please apply it.
I don't have commit rights to the Cactus flesh. I'm going to modify the patch anyway, so please don't commit it just yet.
Do you want to extend it to also auto-generate a makefile definition HAVE_CAPABILITY_XXX?
Good idea. I had thought this was there already, but now I realise that this is done manually in the HDF5.sh script. I will modify the patch so that we have both makefile definitions and preprocessor macros definitions.
I think all we need is to rename cctki_Capabilities.h to cctk_Capabilities.h, and protect it from make the same way cctk_Arguments.h is protected. Then each thorn can include <cctk_Capabilities.h>, and will then automatically include its own capabilities file. Adding/removing thorns won't matter because the file is protected. If a thorn is added that provides a new capability that thorn T uses, then thorn T's file Thorns/cctki_T.h will be modified, and make will recompile the thorn. (This is exactly the same mechanism as for cctk_Arguments.h, and it works fine there.)
Of course! I was confused about this, but you're right, it should work OK that way. Given that it is safe for all thorns to include cctk_Capabilities.h, should we just include it in cctk.h for all thorns?
On Tue, Nov 30, 2010 at 3:19 PM, Ian Hinder ian.hinder@aei.mpg.de wrote:
[schnipp]
Of course! I was confused about this, but you're right, it should work OK that way. Given that it is safe for all thorns to include cctk_Capabilities.h, should we just include it in cctk.h for all thorns?
Yes, I think this would be good. I don't see any real advantage in introducing another user-level include file (but this is a question of taste).
-erik
On 30 Nov 2010, at 21:21, Erik Schnetter wrote:
On Tue, Nov 30, 2010 at 3:19 PM, Ian Hinder ian.hinder@aei.mpg.de wrote:
[schnipp]
Of course! I was confused about this, but you're right, it should work OK that way. Given that it is safe for all thorns to include cctk_Capabilities.h, should we just include it in cctk.h for all thorns?
Yes, I think this would be good. I don't see any real advantage in introducing another user-level include file (but this is a question of taste).
The attached patch implements the things we have discussed. An appropriate commit message would be:
Improve optional capability requirement
* Introduce makefile and preprocessor variables HAVE_CAPABILITY_<cap> for each provided capability
* Remove incorrect definition line
* Rename cctki_Capabilities.h to cctk_Capabilities.h and exclude it from dependency checking (dependencies of the files included from this one will be sufficient) <<<
users@lists.einsteintoolkit.org
-
Erik Schnetter -
Ian Hinder