idoor do
2018-11-14 15:53:48 UTC
Could somebody help me with this issue ? I have been stuck on this issue
for a couple of days.
Thanks
I installed drill-logfile-plugin-1.0.0 JAR file to
<drill_install>/jars/3rdParty/ directory, and configured dfs as the
following, but I got error: "Please retry: error (invalid JSON mapping)",
in the sqlline.log file, it shows an error: Unable to find constructor for
storage config named 'log' of type
'org.apache.drill.exec.store.log.LogFormatPlugin$LogFormatConfig, but I
double checked the drill-logfile-plugin-1.0.0.jar file is in the
jars/3rdParty folder:
My config for dfs with log plugin suport is:
{
"type": "file",
"connection": "file:///",
"config": null,
"workspaces": {
"root": {
"location": "/",
"writable": false,
"defaultInputFormat": null,
"allowAccessOutsideWorkspace": false
},
"test": {
"location": "/Users/tsd",
"writable": false,
"defaultInputFormat": null,
"allowAccessOutsideWorkspace": false
},
"tmp": {
"location": "/tmp",
"writable": true,
"defaultInputFormat": null,
"allowAccessOutsideWorkspace": false
}
},
"formats": {
"log" : {
"type" : "log",
"extensions" : [ "log" ],
"fieldNames" : [ "date", "time", "pid", "action", "query" ],
"dataTypes" : [ "DATE", "TIME", "INT", "VARCHAR", "VARCHAR" ],
"dateFormat" : "yyMMdd",
"timeFormat" : "HH:mm:ss",
"pattern" : "(\\d{6})\\s(\\d{2}:\\d{2}:\\d{2})\\s+(\\d+)\\s(\\w+)\\s+(.+)",
"errorOnMismatch" : false
}
},
"enabled": true
}
If I configured the log section as this to remove some fields, the error
will disappear, but some fields will be missing, and the query:
select * from `mysql.log` limit 10; returns error: ERROR
o.a.calcite.runtime.CalciteException -
org.apache.calcite.sql.validate.SqlValidatorException: Object 'mysql.log'
not found
and when I type show files;, it shows the mysql.log file is in the /Users/tsd
directory:
"log": {
"type": "log",
"extensions": [
"log"
],
"fieldNames": [
"date",
"time",
"pid",
"action",
"query"
],
"pattern": "(\\d{6})\\s(\\d{2}:\\d{2}:\\d{2})\\s+(\\d+)\\s(\\w+)\\s+(.+)"
}
for a couple of days.
Thanks
I installed drill-logfile-plugin-1.0.0 JAR file to
<drill_install>/jars/3rdParty/ directory, and configured dfs as the
following, but I got error: "Please retry: error (invalid JSON mapping)",
in the sqlline.log file, it shows an error: Unable to find constructor for
storage config named 'log' of type
'org.apache.drill.exec.store.log.LogFormatPlugin$LogFormatConfig, but I
double checked the drill-logfile-plugin-1.0.0.jar file is in the
jars/3rdParty folder:
My config for dfs with log plugin suport is:
{
"type": "file",
"connection": "file:///",
"config": null,
"workspaces": {
"root": {
"location": "/",
"writable": false,
"defaultInputFormat": null,
"allowAccessOutsideWorkspace": false
},
"test": {
"location": "/Users/tsd",
"writable": false,
"defaultInputFormat": null,
"allowAccessOutsideWorkspace": false
},
"tmp": {
"location": "/tmp",
"writable": true,
"defaultInputFormat": null,
"allowAccessOutsideWorkspace": false
}
},
"formats": {
"log" : {
"type" : "log",
"extensions" : [ "log" ],
"fieldNames" : [ "date", "time", "pid", "action", "query" ],
"dataTypes" : [ "DATE", "TIME", "INT", "VARCHAR", "VARCHAR" ],
"dateFormat" : "yyMMdd",
"timeFormat" : "HH:mm:ss",
"pattern" : "(\\d{6})\\s(\\d{2}:\\d{2}:\\d{2})\\s+(\\d+)\\s(\\w+)\\s+(.+)",
"errorOnMismatch" : false
}
},
"enabled": true
}
If I configured the log section as this to remove some fields, the error
will disappear, but some fields will be missing, and the query:
select * from `mysql.log` limit 10; returns error: ERROR
o.a.calcite.runtime.CalciteException -
org.apache.calcite.sql.validate.SqlValidatorException: Object 'mysql.log'
not found
and when I type show files;, it shows the mysql.log file is in the /Users/tsd
directory:
"log": {
"type": "log",
"extensions": [
"log"
],
"fieldNames": [
"date",
"time",
"pid",
"action",
"query"
],
"pattern": "(\\d{6})\\s(\\d{2}:\\d{2}:\\d{2})\\s+(\\d+)\\s(\\w+)\\s+(.+)"
}